Hello friends! at times during automation, we are required to fetch all the links present on a webpage. Also, this is one of the most frequent requirements of web-scrapping. In this tutorial, we will learn to fetch all the links present on a webpage by using tagname locator.
If you have a basic understanding of HTML, you must be aware of the fact that all hyperlinks are of type anchor tag or ‘a’.
<a href="selenium-introduction.html">Selenium Introduction</a>
Content
How to fetch all the links on a webpage?
- Navigate to the desired webpage
- Get list of WebElements with tagname ‘a’ using driver.findElements()-
List<WebElement> allLinks = driver.findElements(By.tagName(“a”)); - Traverse through the list using for-each loop
- Print the link text using getText() along with its address using getAttribute(“href”)
System.out.println(link.getText() + ” – ” + link.getAttribute(“href”));
Sample Code to Scrape Links
package seleniumTutorials;
import java.util.List;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
public class GetAllLinks {
public static void main(String[] args){
WebDriver driver = new FirefoxDriver();
//Launching sample website
driver.get("https://artoftesting.com/sampleSiteForSelenium");
driver.manage().window().maximize();
//Get list of web-elements with tagName - a
List<WebElement> allLinks = driver.findElements(By.tagName("a"));
//Traversing through the list and printing its text along with link address
for(WebElement link:allLinks){
System.out.println(link.getText() + " - " + link.getAttribute("href"));
}
//Commenting driver.quit() for user to easily verify the links
//driver.quit();
}
}
Hi Kuldeep, in the above sample code, if I want to print the below format
System.out.println(index value of the link +”:”+ linktext), how can i do it?
Hi Chandra,
Try below lines for expected result
for(int i=0;i<allLinks.size();i++){
System.out.println(i +" : "+ allLinks.get(i).getText());
}
Thank you so much.
List jobLinks =driver.findElements(By.tagName(“loc”));
List texts = new ArrayList();
for (WebElement element : jobLinks) {
texts.add(element.getText());
}
for(String text:texts) {
What if I want to click and open the link
There are different ways to do this. You can store all the links and then navigate to each URL.
List jobLinks =driver.findElements(By.tagName(“loc”));
List texts = new ArrayList();
for (WebElement element : jobLinks) {
texts.add(element.getText());
}
for(String text:texts) {
if I want handle one of the link so which code is work
How to validate title of each link after link get opened in new tab
How do I ignore random links and only want to get certain links from a div?
You have to write a more precise locator instead of just the ‘anchor’ tag.
how to check for 404 in footer ,and also how to naviagte back from other domain to main domain
Thankyou so much , its very clear and easy to understand..
Hi,
Can you help me how to write code.
There are four links in columnA and I want to click each link one after the another .
I tried getting all the links from the column A and able to get text but unable to click on links
Please help me
If you have the link, you can do just driver.get(url)