Both of these companies offer web data infrastructure such as proxies and automated data collection. However, finer details such as pricing and data outputs can differ greatly. They also offer SERP and web unlocking tools to make automated web access easy. In this piece, we’ll go over their offerings and even demo their automated scrapers.
By the end of this article, you’ll be able to answer the following questions.
- What is web data infrastructure?
- What solutions are available for data collection?
- What do infrastructure pricing models actually look like?
- What do outputs from Decodo and Oxylabs look like in the real world?
Decodo
Decodo (formerly Smartproxy) is primarily known for its proxy infrastructure and Site Unblocker (a web unlocking tool). However, they offer automated scrapers as well. These scrapers cover a variety of target sites ranging from eCommerce products to social media and even YouTube.
Proxies
- Residential proxies: Proxies tied to real residential IP addresses. These proxies are often slower and less stable. However, they use a real home network. This makes them very reliable with high-difficulty sites.
- ISP proxies: Internet Service Provider (ISP) proxies are newer to the industry. Using ISP proxies, providers give you access to datacenter hardware while using a residential IP address.
- Mobile proxies: Similar to residential proxies, mobile proxies route your traffic through a real user device. Instead of using a home network, mobile proxies use a mobile network. These proxies are often slower and less stable but help give access to some of the most difficult sites.
- Datacenter proxies: Datacenter proxies are the cheapest option of the bunch. However, these proxies are not inherently bad. For low-medium difficulty sites, datacenter proxies give you the fast and stable access.
- Site Unblocker: Site Unblocker is Decodo’s managed proxy option. Tools like Site Unblocker automatically create your proxy connection using the best available proxy. Site Unblocker also comes with JavaScript support and a 100% success rate guarantee.
Scrapers
- eCommerce Scraping API: Scrape products from sites such as WalMart, Amazon, eBay, etsy and more.
- Web Scraping API: Get web pages from almost any target site and output the data using HTML, JSON, CSV or Markdown.
- SERP Scraping API: Get instant results from search engines such as Bing, Google, Yahoo and Baidu.
- Social Media Scraping API: Extract detailed information from social media sites like TikTok and Reddit.
AI data
- Video downloader: Download video and audio data from YouTube at scale. This helps teams create custom multimodal datasets.
- AI Parser: Simply enter a URL and describe the data you need and output structured data. This feature is available to all users with no added cost.
Oxylabs
Oxylabs was founded by Tesonet — the same parent company that acquired Decodo. They have similar offerings with some important distinctions. They also offer dedicated ISP and datacenter proxies. Unblocking Browser is also available as a separate product for teams who need to render a real browser within the cloud. They also offer ready-to-use video datasets.
Proxies
- Residential proxies: Route your traffic through a real home network. Once again, these connections are often slower and less stable than datacenter proxies.
- ISP proxies: Use datacenter-grade hardware on a residential ISP.
- Dedicated ISP proxies: Use a dedicated IP address from a residential ISP network.
- Mobile proxies: Run your traffic through a mobile service provider. Once again, these connections make it difficult to spot automated traffic but are less reliable and often slower than standard residential proxies.
- Datacenter proxies: Route your internet traffic through quality hardware with a datacenter IP address. Performance is high but accessing difficult sites can come with challenges.
- Dedicated datacenter proxies: Control a dedicated IP address using a datacenter proxy.
- Web Unblocker: Automated proxy selection and site access. Worry about code and let this solution take care of proxy connections and access difficulties.
- Unblocking Browser: A proxy-powered web browser that runs in the cloud.
Scrapers and scraping solutions
- Web Scraper API: A universal scraping solution. They offer prebuilt scraping options for many popular sites like Amazon, Google, ChatGPT and WalMart.
- OxyCopilot: Generate full web extraction scripts by prompting an AI model.
AI data
- Video Data API: Perform multimodal data extraction.
- Video datasets: Get curated datasets of video data.
Decodo
Decodo Scraping API
Now, let’s walkthrough the Decodo Scraping API. Simply create an account and you’ll be able to use their playground. As you can see, you can enter a target site, toggle JavaScript support, tweak your location information and choose from a variety of output formats.
In the image below, we sent a test request to Books to Scrape. We set the output to Markdown. Within a couple seconds, we received the target page.
If you want to look at the output more closely, we have a small portion of it in the snippet below. As you can see, the page has been successfully converted to Markdown. This is a great feature to use when you’re working with AI models. Markdown drastically reduces page size and helps save on your overall token input costs.
[Books to Scrape](index.html) We love being scraped!
* [Home](index.html)
* All products
* [Books](catalogue/category/books_1/index.html)
+ [Travel](catalogue/category/books/travel_2/index.html)
+ [Mystery](catalogue/category/books/mystery_3/index.html)
+ [Historical Fiction](catalogue/category/books/historical-fiction_4/index.html)
+ [Sequential Art](catalogue/category/books/sequential-art_5/index.html)
+ [Classics](catalogue/category/books/classics_6/index.html)
+ [Philosophy](catalogue/category/books/philosophy_7/index.html)
+ [Romance](catalogue/category/books/romance_8/index.html)
+ [Womens Fiction](catalogue/category/books/womens-fiction_9/index.html)
+ [Fiction](catalogue/category/books/fiction_10/index.html)
+ [Childrens](catalogue/category/books/childrens_11/index.html)
+ [Religion](catalogue/category/books/religion_12/index.html)
+ [Nonfiction](catalogue/category/books/nonfiction_13/index.html)
+ [Music](catalogue/category/books/music_14/index.html)
+ [Default](catalogue/category/books/default_15/index.html)
Pricing
Decodo offers a large variety of products. The list below isn’t comprehensive but should give you a solid picture of what your pricing will look like.
- Residential proxies: At the “Regular” level, pricing normally ranges from $6/GB on the 2GB plan all the way to $4.50/GB on the 100GB plan.
- ISP proxies: These begin with a 1GB plan — normally priced at $6.50/GB. Their highest tier is the 250GB plan which costs $1.80/GB.
- Mobile proxies: Mobile proxies begin at $7.50/GB for the 2GB plan. On the 100GB plan, they cost $2.75/GB.
- Datacenter: Datacenter proxies come with much larger bandwidth and cheaper data. Their lowest tier is the 10GB package which costs only $0.60/GB. At the high end, they offer a 1000GB package for $0.45/GB.
- Site Unblocker: Starts at $10.00/GB for the 1GB plan. At scale, teams can use the 100GB plan for $6.75/GB. Teams can also use a per request model which starts at $1.25 per request for 23,000 requests. At the high end, teams get 950,000 requests for $1.05/request.
- Scraping API: Compared to Site Unblocker, Scraping API offers some strong savings when you scale. They begin with the same 23,000 request package for $1.25/request. However, high usage teams pay $0.70/request on the 2,000,000 requests package.
Oxylabs
Oxylabs Web Scraper API
Now, we’ll try the Oxylabs Web Scraper API. You can find it under the “Scraping solutions” section. Click on the Start free trial button to get started. They’ll give you a free trial with up to 2,000 results. No credit card is required.
Now, you’ll be prompted to create a username and password. You can generate passwords randomly for better security. When you’re finished, click on Create API user.
Once it’s finished, make sure to back up your password. You’ll need this password to access your account. From the dashboard, scroll down to “Run your first request”. Click on the Quick start button.
You can send a basic request using cURL.
curl 'https://realtime.oxylabs.io/v1/queries' \
--user "<your-username>:<your-password>" \
-H "Content-Type: application/json" \
-d '{
"source": "amazon_product",
"query": "B07FZ8S74R",
"geo_location": "90210",
"parse": true
}'
Next, head over to the API playground. Once again, we’ll use Books to Scrape as a target site.
The playground does not give us the option to convert the page to Markdown. However, it does provide us with access to Oxylabs CoPilot to create various scripts through LLM prompting. If you look below, you can see our output. We get a raw HTML dump on the left side and we get a preview of the page on the right.
If you need Markdown or other custom outputs, you’ll need to request it from your programming environment. In the example below, we use a simple Python script to retrieve the Markdown content.
import requests
# Structure payload.
payload = {
'source': 'universal',
'url': 'https://books.toscrape.com',
'markdown': True
}
# Get response.
response = requests.request(
'POST',
'https://realtime.oxylabs.io/v1/queries',
auth=('<your-username>', '<your-password>'), #Your credentials go here
json=payload,
)
# Instead of response with job status and results url, this will return the
# JSON response with results.
print(response.text)
We get a JSON response. Our content field looks pretty similar to the output we received from Decodo. However, users should note that in this case, we receive newline characters like \n inside the output — this could cause minor formatting issues and some teams will need to clean the output.
[Books to Scrape](index.html) We love being scraped!\n\n* [Home](index.html)\n* All products\n\n* [Books](catalogue/category/books_1/index.html)\n + [Travel](catalogue/category/books/travel_2/index.html)\n + [Mystery](catalogue/category/books/mystery_3/index.html)\n + [Historical Fiction](catalogue/category/books/historical-fiction_4/index.html)\n + [Sequential Art](catalogue/category/books/sequential-art_5/index.html)\n + [Classics](catalogue/category/books/classics_6/index.html)\n + [Philosophy](catalogue/category/books/philosophy_7/index.html)\n + [Romance](catalogue/category/books/romance_8/index.html)\n + [Womens Fiction](catalogue/category/books/womens-fiction_9/index.html)\n + [Fiction](catalogue/category/books/fiction_10/index.html)\n + [Childrens](catalogue/category/books/childrens_11/index.html)\n + [Religion](catalogue/category/books/religion_12/index.html)\n + [Nonfiction](catalogue/category/books/nonfiction_13/index.html)\n + [Music](catalogue/category/books/music_14/index.html)\n + [Default](catalogue/category/books/default_15/index.html)\n + [Science Fiction](catalogue/category/books/science-fiction_16/index.html)\n
Pricing
Due to the variety of their products, Oxylabs’ pricing plans are pretty expansive as well. Once again, this list is not exhaustive but it should provide teams with a solid picture of what the budget looks like.
- Residential proxies: Pay-as-you-go covers up to 50GB with a standard price of $8/GB. At the top tier, they offer up to 86GB for $3.49/GB.
- ISP proxies: ISP proxies do not offer bandwidth based pricing. Users pay per IP. On the low end, teams pay $1.60/IP address per month. Premium users pay $1.20/IP address per month.
- Dedicated ISP proxies: Low usage plans cost $3.20/IP address per month. At scale, teams pay $2.20/IP address per month.
- Datacenter proxies: Users can access up to five IP addresses per month for free. 10 IPs costs $1.20/IP address. When using over 200 IP addresses per month, teams only pay $0.90/IP address.
- Dedicated datacenter proxies: This one comes at rate of $2.25/IP address for low usage. At 1,000 IP addresses, teams pay $1.20/IP address.
- Mobile proxies: Pay-as-you-go normally costs $9/GB. At their highest tier, teams get up to 80GB for $7.50/GB.
- Web Scraper API: Teams get up to 2,000 requests for free. The Micro plan includes up to 98,000 results at a price of $49/month. At the top tier, the Advanced plan offers 622,500 results at $249/month.
- Unblocking Browser: Plans start at $6/GB for 50GB. At 300GB, teams pay $4.70/GB.
- Web Unblocker: Teams get free access up to 1GB. Paid plans start at $9.40/GB for up to 8GB. At 88GB, teams pay $7.50/month.
Key breakdown: Decodo vs. Oxylabs
| Category | Decodo | Oxylabs |
|---|---|---|
| Residential proxies | Yes | Yes |
| ISP proxies | Yes | Yes |
| Dedicated ISP proxies | No | Yes |
| Mobile proxies | Yes | Yes |
| Datacenter proxies | Yes | Yes |
| Dedicated datacenter proxies | No | Yes |
| Managed web unlocking | Site Unblocker | Web Unblocker |
| Browser-based scraping | No | Yes |
| Scraping API availability | Yes | Yes |
| Prebuilt scraper targets | eCommerce, SERP, web, social media | eCommerce, SERP, web and other popular targets |
| JavaScript rendering | Supported | Supported |
| AI-oriented tooling | AI Parser, video downloader | OxyCopilot, Video Data API, datasets |
| Video data support | YouTube video and audio downloads | Video Data API and curated datasets |
| Pricing model (proxies) | Primarily per-GB | Mix of per-GB and per-IP |
| Pricing model (scraping) | Per-request | Per-result |
Conclusion
Both Decodo and Oxylabs provide web data infrastructure. Decodo tends to be a better fit for teams who need a solid balance between price and infrastructure at scale. Oxylabs supports highly granular control but often at a higher price point with much higher costs for teams needing to scale.
When you choose a web data infrastructure provider, you need to think about the following questions.
- Do they offer automated scraping?
- How easy is integration?
- What does their pricing look like in startup mode and at scale?
Decodo is a solid option for teams looking to maximize throughput as their project scales. Oxylabs tends to be stronger for teams who need features like OxyCopilot, Unblocking Browser or dedicated IP addresses.