Skip to main content
Tech Radar
  • Tech Radar Pro
  • Tech Radar Gaming
Tech Radar Pro TechRadar IT Insights for Business
Subscribe
RSS
Asia
flag of Singapore
Singapore
Europe
flag of Danmark
Danmark
flag of Suomi
Suomi
flag of Norge
Norge
flag of Sverige
Sverige
flag of UK
UK
flag of Italia
Italia
flag of Nederland
Nederland
flag of België (Nederlands)
België (Nederlands)
flag of France
France
flag of Deutschland
Deutschland
flag of España
España
North America
flag of US (English)
US (English)
flag of Canada
Canada
flag of México
México
Australasia
flag of Australia
Australia
flag of New Zealand
New Zealand
Technology Magazines
Technology Magazines
Why subscribe?
  • The best tech tutorials and in-depth reviews
  • Try a single issue or save on a subscription
  • Issues delivered straight to your door or device
From$12.99
View
  • News
  • Reviews
  • Features
  • Expert Insights
  • Website builders
  • Web hosting
  • Security
Trending
  • Best standing desk deals
  • Best cloud storage 2023
  • What is Microsoft Teams?
  • Expert Insights

When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.

  1. Home
  2. Features
  3. Computing
Supported

What is a rotating proxy and how does it work?

By Jonas P. DeMuro
published 31 August 2022

Understanding proxies, and how to leverage them to the task of web scraping

In Association with

Iproyal logo
An abstract image of rotating colors.
(Image credit: Pixabay)

As most folks have heard by now, going online is hardly an anonymous activity. This is because we are continuously tracked, including via our Internet Protocol Address. This IP Address can reveal multiple pieces of information about the user, including their internet service provider, and their geographic location. It can also be utilized to block a user from additional activity when they revisit a website.

However, if an IP address is used for a nefarious activity, it can be blacklisted in a DNS-based anti-spam database. An activity such as spamming could get an IP address entered into such a database.

Protect your online privacy with IPRoyal

Protect your online privacy with IPRoyal

Use TECHRADAR10 at checkout and get an extra 10% off IPRoyal's residential proxies. This discount stacks with IPRoyal's existing bulk discounts.

View Deal

Types of addresses 

IP addresses get divided into two types: static and dynamic. A static IP address, also known as a fixed IP address or a sticky IP address, means that the location gets permanently assigned, and therefore this address does not subsequently change. This is useful when other devices need to connect, such as a network print server, a VPN or a web server.

However, not everything needs a static IP address, and this is why you can use a dynamic IP address. Your average home user just wants the devices to connect easily, so routers use the Dynamic Host Configuration Protocol (DHCP), which assigns IP addresses to client devices as they are needed. Also, each time a broadband modem gets rebooted, the Internet Service Provider assigns a new dynamic IP address from its pool of addresses. 

In general, these dynamic IP addresses are used in this application due to cost considerations, as this way the ISP can just have an available pool of addresses, and then assign them to the users as needed. Therefore, most residential users get a dynamic IP address from their broadband provider.

Understanding the issue 

There are times that a user needs to protect their IP address, such as to protect their anonymity, or to bypass a geo-restriction. The method to typically accomplish this is via a VPN. Via this method, the user puts their information through a proxy server, which then anonymizes them, so that it goes back to the VPN server, and not to the individual’s IP address. However, in this situation, the user gets assigned a single, dynamic IP address from the pool of IP addresses that the VPN assigns as needed, which is the same for the session.

There are times when a user needs not just one IP address, but rather an entire pool of addresses to be able to use. This can be done, but it would require the user to reboot the broadband modem each time you wanted to generate a new address, or to initiate a new VPN session, which would be impractical beyond restarting beyond a few sessions.

Rotating around 

Enter the rotating proxy, which is designed to be able to offer an entire pool of IP addresses, so a user can rotate between the addresses. The actual number varies by the provider, but we are talking about in excess of 100 million from a single provider in some cases. The idea is that there is a large IP proxy pool, and then each time a new connection gets made, the proxy server provider rotates to a new address. Therefore, the user then appears to be a different client to the website. 

Why, oh why? 

The question to understand is what is the intended use of a rotating proxy as the average user will typically simply connect to the site, and won’t need such a large pool of addresses. The answer lies in a process known as web scraping, which is a method to extract the large amount of data located on a website. After this is done, the data can then be exported into a database, for example to be more useful to the user. 

Scraping for info 

Web scraping is often done to keep track of pricing, or reviews on an e-commerce website. While this information can be done manually, this is generally inefficient and time consuming. Performing web scraping via an automated method will obtain far more information quickly, however there are challenges to overcome including captchas that check if a real human is coming to the site. Furthermore, websites don’t really want to be scraped, so they keep track of the IP addresses, and can block the revisits, hence the need for a rotating proxy, so that further visits are done with a new, unique IP address.

In case you are wondering, yes web scraping is legal. This is because the information is publicly available, there is no law being broken. However, if additional data, not publicly available, is extracted then it is not legal, and can generate a lawsuit.

Conclusion 

Having access to a rotating proxy is certainly a useful tool, that can glean much useful data from websites through the process of web scraping.  

We've featured the best business VPN.

TechRadar Pro created this content as part of a paid partnership with Iproyal. The contents of this article are entirely independent and solely reflect the editorial opinion of TechRadar Pro.

Are you a pro? Subscribe to our newsletter

Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!

By submitting your information you agree to the Terms & Conditions and Privacy Policy and are aged 16 or over.
Jonas P. DeMuro
Jonas P. DeMuro
Social Links Navigation

Jonas P. DeMuro is a freelance reviewer covering wireless networking hardware.

More about computing
iTunes in Windows 11

Apple iTunes has a serious security flaw you really should know about

Hologram of security padlock operating on the electronic circuit CPU.

MOVEit Transfer has a major security issue - here's what you need to know

Latest
iPhone showing an Amazon logo on screen

Amazon may give Prime customers a cheaper monthly cellular plan

See more latest ►
Most Popular
8 Meta Quest Gaming Showcase 2023 games Oculus Quest 2 players must see

By Hamish HectorJune 01, 2023

If ChatGPT does become sentient, it's smart enough to hide it from us

By Muskaan SaxenaJune 01, 2023

Telltale's The Walking Dead only came alive when I played it with friends

By Julian BensonJune 01, 2023

Drummer is “more vulnerable, more open” in The Expanse: A Telltale Series

By Julian BensonJune 01, 2023

The Expanse: A Telltale Series needs more gravity

By Julian BensonJune 01, 2023

Bluetooth Auracast explained: 3 big reasons to upgrade your wireless headphones

By Becky ScarrottJune 01, 2023

7 great new horror movies to stream on Netflix, Prime Video, Hulu and more in June 2023

By Louise BlainJune 01, 2023

MSI branches out from its gaming roots at Computex 2023

By John LoefflerJune 01, 2023

Blasphemous 2’s ritual punishment will bring you to your knees

By Ryan LestonMay 31, 2023

Diablo 4’s multiplayer is where the real game begins

By Cat BussellMay 31, 2023

One of Overwatch 2’s oldest heroes is out of the closet, but is this enough?

By Elie GouldMay 30, 2023

  1. Cooler Master Computex 2023
    1
    7 signs Cooler Master has gone bonkers at Computex 2023, and that’s a good thing
  2. 2
    Windows 11 Moment 3 update is packed with cool features – here are 5
  3. 3
    How to watch The Ultimate Fighter season 31 online: stream the MMA reality show
  4. 4
    Audio Editing simplified with Wondershare Filmora AI
  5. 5
    6 new Netflix Original movies and shows you can’t miss in June
  1. A young AI-created humanoid touches a robot in The Creator
    1
    The Creator looks like The Last of Us, Terminator, and Star Wars rolled into one
  2. 2
    The iPhone and Galaxy S23 Ultra prove money is no object for smartphone buyers
  3. 3
    New iPhone 15 Pro Max specs leak hints at what's not changing this year
  4. 4
    5 upcoming TV shows that mean Apple TV Plus could replace your Netflix subscription
  5. 5
    Drummer is “more vulnerable, more open” in The Expanse: A Telltale Series
Technology Magazines
  • ●
Technology Magazines
  • The best tech tutorials and in-depth reviews
From$12.99
View

TechRadar is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site.

  • About Us
  • Contact Future's experts
  • Contact Us
  • Terms and conditions
  • Privacy policy
  • Cookies policy
  • Advertise with us
  • Web notifications
  • Accessibility Statement
  • Careers

© Future US, Inc. Full 7th Floor, 130 West 42nd Street, New York, NY 10036.