How To: Save map documents in ArcGIS 10.x to previous version of ArcGIS in batch

Summary

Instructions provided describe how to use a sample script to batch save map documents in ArcGIS 10.x as earlier document versions of ArcGIS.

Procedure

After migrating to another version of ArcGIS, older map documents are upgraded to the installed version the next time they are saved. To share the map documents with older ArcGIS clients, use the Save A Copy command from the ArcMap File menu to save each map document.

However, if there are many map documents involved, a Python script tool can be used to batch convert. Apply the following process to avoid opening each document.

Below is a sample Python script for saving map documents as earlier document versions of ArcGIS.

Follow us on:

youtube

Twitter

Google+

subscribe to our channel

#######################################
import arcpy, sys, os, string

mxdList = string.split(arcpy.GetParameterAsText(0), ";")
outloc = arcpy.GetParameterAsText(1)
suffix = "_10.2"
version = arcpy.GetParameterAsText(2)
for item in mxdList:
item = item.strip('\'')
mxd = arcpy.mapping.MapDocument(item)
base = os.path.basename(item)
base = os.path.splitext(base)[0] + suffix + os.path.splitext(base)[1]
mxd.saveACopy(outloc + os.sep + base, version)
arcpy.AddMessage(os.path.basename(item) + " has been converted")
##########################################

1. To run this sample script as a tool, paste the code above into a text editor and save as a .py file.
2. Create a new custom toolbox.
3. Right-click the toolbox and select Add > Script.

   Add A New Script Tool

   
   

Follow us on:

youtube

Twitter

Google+

subscribe to our channel

4. In the wizard, name the tool, and click Next.

   Name the Script Tool

   
   

5. Navigate to the location of the .py file.

   Browse to Python Script File

   
   

6. Set the parameters for the Input MXDs and the Output Location. For the Input MXDsparameter, set MultiValue to Yes.

   Set The First Two Parameters As Follows

   
   

7. Set the parameter for the Output Version and add an Input List for the different versions so that the newest version in the list matches the version of the software. Click Finish.

   Note:
   Using the version '10.1' option from version 10 does not work. The function can only save to the current or previous versions.

    

   Setup a Version List to Choose From

   
   

8. Double-click the Script tool to open and run it. Specify the input MXDs, output location, and output version. Click OK to run.

   Open the Tool and Run It

   
   

How to Build a Website From Scratch (With Screenshots)

It’s never been easier to create a website than it is now. There’s an incredibly wide variety of free tools, affordable premium ones, competitive hosting and overall easy to use software to get a site up and running in less than an hour.

On the flipside, this means that more sites are springing up than ever. For this reason, knowing the basics on how make a proper website from scratch is more important than ever.

Find the right webhost and domain registrar

Web hosts are companies that provide technology and storage for your website. Essentially, they “host” your files and website domain on their servers, which they then display to visitors on your website.

The online hosting industry is huge and has both big and small players so there is no shortage to choose from.

However, there are a few things you should look out for when picking a webhost.

• Bandwith and processing resources they provide.
• How much storage space you have.
• Overall reputation.
• How quickly they respond to customer complaints.
• Price

Good processing resources and storage are the most important aspects to take into consideration, since they can speed up your site considerably.

A slow website is bad for business because visitors can get frustrated and press the “Back” button before the page loads. This in turn impacts your SEO, meaning your site won’t rank as well on search results. In practical terms, a slow site decreases your visitors, sales and email subscribers.

Another important criterion is how quickly they respond to customer complaints. In the unfortunate event of a hack or technical downtime, you really want your hosting provider to be quick and get your site up and running as soon as possible.

When it comes to price, we strongly suggest not to be tempted by “free hosting providers”. They have to earn money somehow, so they most likely cut corners, severely limit your resources or worse: they sell your information or feed you ads.

Hosting with SiteGround

Out of the many webhosts out there, SiteGround is one of the best (if not the best). Both in terms of features they offer as well as trust and reliability. As a testament to their quality, WordPress has endorsed them as one of the three trusted web hosts they recommend.

Here’s what makes SiteGround a good choice for someone who sets out to make a website from scratch:

• Excellent 24/7 customer support.
• A malware scanner for your website.
• Free SSL certificates for your website. (important for better rankings)
• One of the fastest webhosts out there.

To begin the process of registering your website, simply go to https://www.siteground.com/web-hostingand choose the plan that best fits your needs.

Our personal recommendation is to go for the GrowBig plan, even if it’s on the pricier side compared to the StartUp plan, since it offers you some really awesome benefits:

1. Host unlimited websites.
2. 20GB storage space.
3. Website backup and restore tool.

You can’t go wrong however, regardless of the plan you choose.

Adding a domain name

An internet domain is essentially the name of your website.  In our case it’s “extendthemes.com”.

Registering a domain name is a distinct process from acquiring a hosting provider. However, most web hosting companies also offer to register your domain name for you once you acquire hosting.

Once you’ve decided on SiteGround, or another hosting provider, the next step is to connect a domain name to the host’s servers. This can be done in two ways, depending on whether or not you’ve already registered your domain name.

You don’t have a domain name

If you haven’t yet registered a domain name, you can do so directly from SiteGround once you’ve pressed on the order button.

Website names are important, both for branding and SEO purposes, so you should put some thought into it and scour the web for a name that is both expressive and available.

Here are just a few things you should consider:

• Do you want the keyword into your business name? For instance, an interesting name for a toyshop might be cooltoys.com. This helps makes your business more easily searchable.
• Length matters. Try to keep it short, if possible under 10 characters. This makes it easier for people to remember and manually type in your website.
• If the name is composed of words, try to use no more than 2.
• Use the right extensions. As a rule of thumb, pick .com or .net if you want to target users across the globe. If visitors target a certain country, choose extensions such as .fr, .de or .co.uk.

Of course, these are just guidelines, and not hard and fast rules. With persistence and a stroke of luck, every site can become successful so don’t obsess over this too much.

You have a registered domain

If you do already have a domain registered, then you will have to change your name servers so they point towards SiteGround. Don’t worry though, there is no “penalty” for doing this.

For instance, you won’t lose anything if you register a domain at GoDaddy and then host it with SiteGround. Millions of websites do this, without any negative effects.

The exact steps you should follow to change nameservers are different from one domain name registrar to another, but in most cases all you need to do is to just copy/paste a few lines from SiteGround’s settings section into the name server section of your registrar.

First, you need to log in to your SiteGround account, go to the My Accounts tab -> Information & Settings. You will find your name servers at the Account DNS section.

Once you know your SiteGround name servers, enter your GoDaddy account and look for “My Products” or “My Domains” sections. The two sections look different, but both will contain your basic website info.

Next, find the “DNS” or “Manage DNS” button. Pressing it will take you to the DNS management menu of your respective site.

Next, under the “Nameservers” section, press the green “Change” button to edit your name servers, and put in the ones from SiteGround.

All you have to do is just copy/paste the nameserver addresses from SiteGround into GoDaddy.

That’s it! At this point, it takes 1-2 days in order for the changes to take effect. This is because name servers have to undergo a process called “propagation” during which the information for your website is updated across the entire Internet.

After propagation is complete, your site is ready to go and you can begin to create the actual website users will interact with.

Choosing a Content Management System

Content Management Systems, or CMS for short, is special software that allows you to create the actual website users interact with. This can be blog posts, photo galleries, eCommerce product listings etc.

There are quite a few such CMS available to a regular user, but in this guide we’ll focus on one in particular: WordPress.

WordPress is the most popular such CMS, with around 55% market share. It’s a very flexible platform which you can use to build just about any kind of site that you want.

The best thing is, users who don’t know how to code can easily set up a beautiful website in just a couple of minutes, by pressing just a few buttons. And thanks to the thousands of themes and plugins available, you can customize your website anyway you want.

Of course, there are a whole bunch of other CMS systems you can use. But most of them come with a series of drawbacks that WordPress either doesn’t have, or can be compensated quickly with a plugin.

Installing WordPress on SiteGround

SiteGround has a very simple installation process for WordPress.

Log in to your SG account, then go to My Accounts and press the Go to cPanel button.

You are now in cPanel, which is the back-end admin console for your website, where you have access to all the features SiteGround can provide.

While cPanel can seem intimidating at first, most website owners rarely need to visit it again past the initial setup. In our case, that means installing WordPress.

To proceed with the installation, press the WordPress button in the Autoinstallers section.

Next up, press “Install Now”, which will take you to a menu where you need to do the finishing touches before completing the installation.

How to manually install WordPress if you’ve chosen another host

If you’ve chosen another host that doesn’t have the quick installation process, then you will have to set up WordPress on your site manually.

Setting up a WordPress website

Congratulations, you’re now on WordPress and can build up your blog, eCommerce shop or any other type of site that you need.

Below is a quick overview of the main tabs on WordPress and what they do. If you want to, skip this step and go directly to the next section on how to install a theme.

Navigating the side menu

Posts are the blog articles of your website. They are different from the Pages section since they are mostly focused around writing articles, whereas Pages are mostly navigational aspects of a website.

Media is where you can access any images, videos and other files you have uploaded into the WordPress admin area. This section groups together both published and unpublished files.

Pages are the organizational elements of a website, such as the “About”, “Contact”, “Disclaimer” pages.

Comments section is where outside visitors leave a reply or give feedback to your pages or blog posts. Unfortunately, most of the comments you will receive will be spam that promises random stuff such as immortality, free money etc.

Appearance is where you actually design and create your website, while also offering access to some of the actual code behind your website (if you so choose to tinker with it).

Plugins section is one of the reasons why WordPress is such a great CMS. In here, you can find software that adds extra functionality to your website, such as contact forms, pop-ups, video players, Google Analytics integrations, cache cleaners and a whole bunch of other features. Because WordPress is so widespread, you can find a plugin for just about anything you want.

Users is where you can see who has admin rights to your WP dashboard, and modify them accordingly.

Tools and Settings allows you to configure aspects of your WP site, such as cache, link structure, redirects, etc.

Choosing a WordPress theme

Themes are essentially predesigned “sites” that you load up into the admin area so users can actually do something once they visit your page.

WordPress has a huge number of free themes available for download and makes it super easy to install one.

First, go to the Appearance section, hover your mouse over it, and press the “Themes” button.

This will take you to a screen that lists all of the themes you have installed on your WordPress website.

WP itself comes with some themes preinstalled. These are fairly functional, good even. But you might want something more specific and with a different look and feel.

Fortunately, there are thousands of themes to choose from, all with their own distinctive designs, catered to the most niche website users.

To start browsing for themes, simply press the Add New button next to the Themes section headline.

This will take you to a separate menu, where you can browse all of the themes available on WordPress.org.

If you want to try out a particular theme, all you have to do is hover your mouse over it and press the Preview button. This will take you to a separate window where you can play around with the theme and see how your website would look like.

If you found a theme you would like to use, simply press the “Install” button, followed by “Activate” if you wish to make it the face of your website.

That’s it! You have now installed a theme on your website, and have started the foray into the online world.

However, there is a chance the vision you have for your website is so niche, no predesigned theme can properly do it justice.

In this case, you might want to try something else: site builder themes.

Build a website with a what-you-see-is-what-you-get WP theme

Some themes, such as Mesmerize FREE, give you a lot more freedom to build a website by providing you with many design elements, which you then use like LEGO’s to design the website you want from scratch.

With site builder themes, you don’t have to worry about some other website that uses the same theme as you and looks like a clone, since you can just design it to be completely unique.

On the flipside, you can also use the flexibility of site builder themes to model your website on some other page that you find attractive / useful.

And best of all, these themes are user friendly and don’t require a lot of time to learn.

Customizing your WordPress installation

By now, you should have all of the following:

• A good hosting provider
• A nice domain name
• An installed WordPress theme.

What comes next is adding the finishing touches to your website and customizing its structure.

How to create pages for your website

Pages are the organizational elements of a website, such as the “About”, “Contact”, “Disclaimer” pages. Their main purpose is to help users navigate around a site to find information which isn’t available on the homepage.

To create a Page, simply go to the WordPress sidebar in the dashboard, and hover the mouse over the Pages tab. This will bring up two options: Add New and All Pages.

Pressing Add New will take you to a screen that lets you edit the Page according to your requirements.

Note 1: As a default, WordPress uses the title of the page as its URL. So, if you title your page as “I haz website”, then the url will look like this: www.yoursite.com/i-haz-website.

Most of the times, you’ll probably be fine with this. But you can edit the link simply by pressing the Editbutton underneath the Title row.

Knowing how to edit links is good skill to have, because proper link structures help your SEO and search rankings.

Note 2: The Parent Page attribute allows you to create hierarchies for your pages. This is relevant because it affects how your pages are ordered.

For instance, if you create a page titled “My Work” and didn’t set a parent page for it, it would have the following URL: www.yoursite.com/my-work.

However, if you decided to use the “I haz website” page as a parent for “My Work”, then the URL would look something like this: www.yoursite.com/i-haz-website/my-work  .

Note 3: Set a Featured Image for your page/blog post.

Featured images are photos or other such media that you can put on top of your blog post or page to make it look cooler or more personalized.

There is no universal standard for how these photos will look. In other words, some themes will change the size or position of this featured image or even remove it altogether.

Adding Pages to Menus

Once you’ve created a Page, you need to make it accessible from the navigation menu, like the one we have on our own page.

To do this, go to the Appearance section of the website, and select Menus.

This takes you to a window that allows you to customize your navigation bar.

From here, you can configure which page goes to which menu. Your window might be a bit different than the one shown here, depending on the theme you chose to install. Overall though, it should be roughly the same process.

Create blog posts

To write a blog post, simply hover over the “Posts” button, and press “Add New”. This will take you to a word editing screen that looks nearly identical to the one in “Pages”.

The Posts section has a few extra sections such as Post Format, Categories and the Publish/Previewbuttons which we won’t cover in this article.

Customize your website

All WordPress themes come with the ability to do customizations and modifications. However, some themes only allow you only basic changes while others let you modify pretty much anything you want.

To customize your site, go to Appearance -> Customize. This will take you to a screen that should look something like this:

The exact customization options vary from theme to theme, so you might want to try out a few more if you want a certain feature.

Widgets

An important element of WordPress sites are widgets. These are special building blocks which contain software that has various functionalities such as:

• List of recent articles.
• Calendar.
• Contact forms.
• Categories

That’s just a sample of what widgets can do. Pairing widgets with the best plugins can add a lot more powerful features to your website.

Most WordPress themes usually display widgets on the side the screen, like this:

THE STEP-BY-STEP GUIDE: GDPR COMPLIANCE FOR WEB SCRAPERS

Unless you’ve been living under a rock for the past few months you know that the EU’s General Data Protection Regulation (GDPR) is upon us.

It is the most comprehensive data protection law ever been introduced, fundamentally changing the way companies can use the personal data of their customers and prospects.

There are countless articles and guides about how GDPR will affect your company’s marketing efforts, lead generation, etc. and the changes you’ll need to make to ensure your company is in full compliance with the law.

But when it comes to web scraping....nothing.

Which is strange given that web scraping has traditionally been the backbone of many companies marketing, lead generation and market intelligence efforts.

To shed some light on this grey area, I sat down with Sanaea Daruwalla, Head of Legal at Scrapinghub, to get her insights on how Scrapinghub ensures our clients are scraping personal data in a GDPR compliant way.

In this guide I will share with you:

• How GDPR is going to impact web scraping (Answer: unless you have the person's explicit consent it is now illegal to scrape a EU residents personal data under GDPR).
• The exact decision making process Sanaea uses when evaluating the GDPR compliance of a web scraping project, and
• Answer the most pressing questions web scrapers have about GDPR compliance.

Before we get started though, I want to highlight a quick disclaimer.

Disclaimer: I am not a lawyer, and the recommendations in this guide do not constitute legal advice. Our Head of Legal is a lawyer, but she’s not your lawyer, so none of her opinions or recommendations in this guide constitute legal advice from her to you. The commentary and recommendations outlined below are based on Scrapinghub’s experience helping our clients (startups to Fortune 100’s) maintain GDPR compliance whilst scraping 7 billion web pages per month. If you want assistance with your specific situation then you should consult a lawyer.

Now with the technicalities out of the way, let’s talk about how you should evaluate your web scraping project for GDPR compliance.

Step 1: Are You Scraping Personal Data?

This is the very first and most obvious question you should be asking yourself when you are instigating a web scraping project.

The General Data Protection Regulation, or GDPR as it is more commonly known, only applies to personal data. Which is defined as any personally identifiable information (PII) that could be used to directly or indirectly identify a specific individual. Examples of personal data include a person's:

• Name
• Physical Address
• Email Address
• Phone Number
• Credit Card Details
• Bank Details
• IP Address
• Date of Birth
• Employment Info
• Social Security Number
• Medical Information
• Video/Audio Recording

If you aren’t scraping personal data, then GDPR does not apply. However, if you are scraping personal data then move to step 2.

Step 2: Are Your Scraping the Personal Data Of EU Residents?

If you are scraping personal data then the next question you need to ask yourself is whether or not you are scraping the personal data of EU citizens or residents (note that the GDPR actually covers the EEA, which includes all EU countries, plus Iceland, Liechtenstein, and Norway, so it’s a bit broader than just the EU).

GDPR is an EEA specific regulation, so it only applies to EU citizens. If you are scraping the personal information of residents of other countries (ex. US, Canada, Australia, etc.) then GDPR may not apply. You just need to comply with the data protection laws in the jurisdiction that you scraping personal data from.

Step 3: Do You Have A Lawful Reason to Scrape Their Personal Data?

Ok, now we are starting to get into the nuts and bolts of GDPR. We now know we are scraping personal data and there will be EU citizens affected. The next question we need to ask yourselves is:

Do we have a lawful reason to scrape the personal data of these EU citizens?

Under GDPR to use or hold the personal data of any EU citizen a company must comply with one or more of the following legal reasons for storing or using their personal data, otherwise they will be in breach of the regulation. The five types of lawful reasons are:

• Consent - the data subject (person whose data we have) consented to us having their data.
• Contract - the personal data is required for performance of a contract with the data subject.
• Compliance - necessary for compliance with a legal obligation.
• Vital Interest, Public Interest, or Official Authority - typically only applicable for state run bodies where access to personal data is in the public's interest.
• Legitimate Interest - necessary for our legitimate interests.

When a client comes to Scrapinghub looking to scrape the personal data of EU residents we take it on a case by case basis because it is vital that you can prove that you have a lawful reason to scrape that data.

The most common legal reasons in the case of web scraping are legitimate interest and consent.

First, let’s take a look at consent...

Consent

For most web scrapers, demonstrating that you have consent from the individual to scrape their personal data will be the main (and often only) method in which you can lawfully scrape the personal data from EU residents.

Prior to the commencement of GDPR, there was a lot of discussion within the web scraping community on whether an EU resident had to implicitly give their consent for companies to scrape their personal data if it was available on public websites (no login required to see the data).

The argument was that by uploading personal data to a public site you are giving consent for that data to be viewed and stored by 3rd parties.

However, after in-depth review of this argument by Sanaea (Head of Legal at Scrapinghub) and external legal experts contracted by Scrapinghub we concluded that this interpretation of the regulations wasn’t compliant with GDPR.

As a result, to scrape the personal data of EU residents you now need to demonstrate that you have the explicit consent of the individual before scraping their personal data.

A lot of web scrapers mightn’t like this position, but after a careful review of all the guidance documents provided by the commission Scrapinghub believes that adopting this policy is the only one that is guaranteed to prevent you and your company falling foul to GDPR.

Obviously, this interpretation of the GDPR regulations will significantly curtail most web scraping projects focused on extraction of the personal information of EU residents for lead generation, market analysis, etc.

However, it will still enable some companies to scrape the personal data of EU citizens if they have obtained their explicit content to do so. An example of this would be companies like Mint.com, where users give Mint consent to log into their online banking accounts and retrieve their banking transactions so that they can be tracked and displayed in a more user friendly format on Mint.com.

Next, we’ll look at using “legitimate interest” as the your lawful reason for scraping the personal data of EU citizens.

Legitimate Interest

The other likely lawful reason available to web scrapers is if they can demonstrate they have a legitimate interest in scraping/storing/using this personal data.

Although this lawful reason is viable for web scrapers, for most companies it will be very difficult for them to demonstrate that they have a legitimate interest in scraping someone's personal data.

In most cases, only governments, law enforcement agencies, etc. will have what would be deemed a to have a legitimate interest in scraping the personal data of its citizens as they will typically be scraping people's personal data for the public good.

Step 4: Assess The Personal Data Being Scraped

As mentioned in Step 3, when a client approaches Scrapinghub looking to scrape the publicly available personal data of EU residents we take it on a case by case basis and work with the client to ensure that this data is being extracted in a GDPR compliant manner.

During this stage not only do we look at the companies lawful reason for scraping personal data we also look at the type of personal data they want to extract, the extent of the proposed data collection and how they plan to use the data post extraction.

There are a number of reasons for taking this approach:

Reason #1: Prevent the Scraping of Sensitive Data

Under the GDPR regulation, there are certain types of data that are classed as “sensitive” . These include any type of personal data that could indicate a person's:

• Racial or ethnic origin
• Political opinions
• Religious or philosophical beliefs
• Trade union membership
• Genetic data
• Biometric data for the purpose of uniquely identifying a natural person
• Data concerning health or a natural person’s sex life and/or sexual orientation

Scraping sensitive data means that you are subject to additional rules and require specific consent to be given for this data to be scraped and stored. Therefore, unless you have clear explicit consent and legitimate reason to scrape this data you should avoid scraping it.

Reason #2: Minimise the Extent of The Data Extraction

A important part of GDPR is that companies should only store and process as much data as is required to successfully accomplish a given task.

Given web scrapings ability to extract large quantities of data from a website there is sometimes the desire to capture as much data as possible as it might be useful in the future. Obviously, this mindset isn’t in line with the new GDPR regulations.

As a result, when Scrapinghub is evaluating a scraping project we often work with client companies to minimise the amount of personal data they extract from a website and to define retention periods to ensure they comply with GDPR. You should adopt a similar evaluation process for your own scraping projects to ensure you comply with GDPR’s minimisation requirements.

Reason #3: Ensure Personal Data Is Used For Legitimate Business Purposes

Even if you can argue that you have a legitimate interest in this data or have the users consent to extract and store their personal data, under GDPR you need to have a clear and legal reason for doing so and be able to demonstrate that it will be used for legitimate business purposes.

If the proposed scraping project doesn’t raise any red flags after being evaluated on these criteria then we will generally commence the scraping project.

Step 5: Ensure The Correct Data Retention And Access Policies Are In Place

As outlined in Step 3, the reason a web scraper is allowed to scrape personal data from a website under GDPR is either because you have their explicit consent or you can demonstrate that you have a legitimate interest in scraping/storing their data.

However, you also need to ensure that the data subjects have been made aware of your data protection and privacy policy and make sure you comply with their Data Subject Access Rights (DSAR), including their right to withdraw consent, request a copy of their data, or request deletion of their data.

If consent is withdrawn, or a DSAR is received to delete personal data, then the company who scraped this data must either delete or anonymize this personal data because you no longer have a legal basis to hold it.

Finally, so your web scraping project is just about ready to go but the last thing you need to check off your list is ensuring your proxies are GDPR compliant, specifically any residential proxies you might be using.

Step 6: Make Sure Your Residential IPs Are GDPR Compliant

As the GDPR regulation defines IP addresses as personally identifiable information you need to ensure that any EU residential IPs you use as proxies are GDPR compliant.

This means that you need to ensure that the owner of that residential IP has given their explicit consent for their home or mobile IP to be used as a web scraping proxy.

If you own your own residential IPs then you will need to handle this consent yourself. However, if you are obtaining residential proxies from a 3rd party provider then you need to ensure that they have obtained consent and are in compliance with GDPR prior to using the proxy for your web scraping project

Old Web Scraping Projects

That is everything you need to know about any future web scraping projects, however, what does GDPR mean for personal data that you may have extracted from websites previously?

Luckily for us you just need to use the same process as outlined above to ensure the GDPR compliance of any old web scraping projects:

1. Audit your databases for any personal data you obtained via web scraping.
2. Determine if this personal data belongs to EU residents.
3. If there is personal data belonging to EU residents then determine if you had a lawful reason for scraping and storing it.
4. If you didn’t have a lawful reason for scraping and storing this data then you must delete or anonymise this personal data.
5. If you did have a lawful reason for scraping and storing this data, then ensure that you’ve put in the adequate data subject access request requirements and retention policies to maintain compliance. .
6. If this data is still publically available, you should determine 1) do you still need all or part of this data to carry out your business processes or 2) if this data contains sensitive data. If you don’t need this data or if it contains sensitive information then delete it.

Conclusion

GDPR is perhaps the most impactful data protection law ever passed, and it will change the way data is extracted from websites forever.

If you are considering commencing a web scraping project for your business that might extract personal data from public websites and you want to ensure it is GDPR compliant, then don’t hesitate to reach out to us. Our engineering team of 60+ crawl engineers and data scientists can build a custom web scraping solution for your specific needs.

f you're interested in web scraping and interested in joining a 100% team of some of the leading web scraping experts then be sure to check out our jobs page. We're growing fast and need people like you to help turn the web into useful data.