Finding broken images in a webpage using Selenium

We were checking for broken URLs using Selenium in the previous post. Now we are going to see how to find broken images in a webpage. Broken images are also called Red-X images. We have some tools in the market for finding red-x images on a website or webpage. For example, InSite which is a windows based application.

The same functionality can be possible with Selenium script. How do we do that?

• We need to find the number of images on the page
• We need to track the properties of each and every image.
• Finally we need to extract the image size amd check if its broken or not

How to find the number of images on the page?

We can find the number using selenium.getXpathCount("//img").intValue() method.

selenium=new DefaultSelenium("localhost", 4444, "*firefox", "http://www.yahoo.com");

selenium.start();

selenium.open("/");

int linkCount = selenium.getXpathCount("//img").intValue();

How to track the properties for each and every image?

We can use the for loop and track the properties of the image one by one.

  for (int i = 1; i < imageCount; i++)

 {

String currentImage = "this.browserbot.getUserWindow().document.images[" + i + "]";

 

}

How to extract the image size from the image property?


String s = selenium.getEval("(!" + currentImage + ".complete) ? false : !(typeof " + currentImage + ".naturalWidth != \"undefined\" && " + currentImage + ".naturalWidth == 0);");
if(s.equals("false"))
{
System.out.println(selenium.getEval(currentImage + ".src"));
}

Find the complete script below:


import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintStream;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.SeleneseTestBase;

public class FindBrokenImages extends SeleneseTestBase{

DefaultSelenium selenium;
public int invaildImg;

@BeforeMethod
public void SetUp()
{
        selenium = new DefaultSelenium("localhost", 4444, "*firefox", "http://www.yahoo.com/");
        selenium.start();
    }

@Test
public void testMethod() throws IOException {

invaildImg=0;
try
{
selenium.open("/");
}
catch (Exception e)
{
selenium.waitForPageToLoad("30000");
}
int imageCount = selenium.getXpathCount("//img").intValue();
FileOutputStream fout = new FileOutputStream ("broken_images.txt", true);
new PrintStream(fout).println("URL : " + selenium.getLocation());
new PrintStream(fout).println("--------------------------------------------");
   for (int i = 1; i < imageCount; i++) {
       String currentImage = "this.browserbot.getUserWindow().document.images[" + i + "]";
       String s = selenium.getEval("(!" + currentImage + ".complete) ? false : !(typeof " + currentImage + ".naturalWidth != \"undefined\" && " + currentImage + ".naturalWidth == 0);");
       if(s.equals("false")){
       // System.out.println(selenium.getEval(currentImage + ".src"));
        new PrintStream(fout).println(selenium.getEval(currentImage + ".src"));
        invaildImg++; }
   }
   new PrintStream(fout).println("Total broken images = " + invaildImg);
   new PrintStream(fout).println(" ");
   fout.close();
 
}

@AfterMethod
public void tearDown()
{
selenium.close();
selenium.stop();
}
}

The above script will identify all the broken images(if any) in yahoo.com and store the image URLs in a notepad file called broken_images.txt. If you want to check the broken images for N number of URLs, you can pass the parameters through Data Provider concept or Excel sheet using JXL package.

Finding the broken links in a webpage using Selenium

We heard about some Firefox plug-ins to find broken links in a webpage, like Link Checker, Xenu and etc. We need to install these plug-ins with Firefox browser and find the broken URLs or 404 pages.

We can write the Selenium script for the same functionality. How can we do that?

• We need to find the number of links available on the page
• We need to track each and every link
• Finally we can get the response code for each and every URL or link with the help of HttpURLConnection Class and getResponseCode method.

How to find the number of links on the page?

We can find the number using selenium.getXpathCount("//a").intValue() method.

selenium=new DefaultSelenium("localhost", 4444, "*firefox", "http://www.yahoo.com");

selenium.start();

selenium.open("/");

int linkCount = selenium.getXpathCount("//a").intValue();

How to track each and every link on the page?

We can use the for loop and track the links one by one using this.browserbot.getUserWindow().document.links[] method. This will return the complete properties of the <a> tag of each URL. Then we can use selenium.getEval() method to extract only the HREF part of the <a> tag.

for (int i = 0; i < linkCount; i++) 

   {

     

       currentLink = "this.browserbot.getUserWindow().document.links[" + i + "]";

       temp = selenium.getEval(currentLink + ".href");

             }

How to find out the response code of the URL?

We can use HttpURLConnection Class and getResponseCode method for finding the reponse code of the URL.

public static int getResponseCode(String urlString) throws MalformedURLException, IOException {

   URL u = new URL(urlString); 

   HttpURLConnection huc =  (HttpURLConnection)  u.openConnection(); 

   huc.setRequestMethod("GET"); 

   huc.connect(); 

   return huc.getResponseCode();

}

Find out the complete Selenium Script below:

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.PrintStream;

import java.net.HttpURLConnection;

import java.net.MalformedURLException;

import java.net.URL;

import org.testng.annotations.BeforeMethod;

import org.testng.annotations.Test;

import com.thoughtworks.selenium.DefaultSelenium;

import com.thoughtworks.selenium.SeleneseTestBase;

public class BrokenURL extends SeleneseTestBase {

public int invalidLink;

String currentLink;

String temp;

public DefaultSelenium selenium;

@BeforeMethod

public void setUp() throws Exception

{

selenium=new DefaultSelenium("localhost", 4444, "*firefox", "http://www.yahoo.com");

selenium.start();

}

@Test

public void testUntitled() throws Exception {

FileOutputStream fout = new FileOutputStream ("broken_links.txt", true);

invalidLink=0;

selenium.open("/");

int linkCount = selenium.getXpathCount("//a").intValue();

   

new PrintStream(fout).println("URL : " + selenium.getLocation());

new PrintStream(fout).println("--------------------------------------------");

   for (int i = 0; i < linkCount; i++) 

   {

    int statusCode=0;

   

       currentLink = "this.browserbot.getUserWindow().document.links[" + i + "]";

       temp = selenium.getEval(currentLink + ".href");

       statusCode=getResponseCode(temp);

       if (statusCode==404)

       {

        new PrintStream(fout).println(selenium.getEval(currentLink + ".href") + " "+ statusCode);

        invalidLink++; 

       }

   }

   new PrintStream(fout).println("Total broken Links = " + invalidLink);

   new PrintStream(fout).println(" ");

fout.close();

   System.out.println(currentLink);

   System.out.println(temp);

}

public static int getResponseCode(String urlString) throws MalformedURLException, IOException {

   URL u = new URL(urlString); 

   HttpURLConnection huc =  (HttpURLConnection)  u.openConnection(); 

   huc.setRequestMethod("GET"); 

   huc.connect(); 

   return huc.getResponseCode();

}

public void tearDown()

{

selenium.close();

selenium.stop();

}

   

}

The above script will identify all the broken links(if any) in yahoo.com and store the 404 URLs in a notepad file called broken_links.txt. If you want to check the broken links for N number of URLs, you can pass the parameters through Data Provider concept or Excel sheet using JXL package.

FitNesse and Selenium

When we create our Selenium tests in Java normally we use JUnit or TestNG frameworks to execute our test scripts. We can also use FitNesse tool as a framework for our Selenium tests. FitNesse can be used when we have the below scenarios:

• When we need to pass number of parameters to the script
• When we need to run the same script with different parameters
• When we need to run the same script with different URLs as parameters (functionality will be same).

You might have this question running in your mind!! Can't we use JUnit or TestNG to achieve the above things. Answer to this question will be, YES, we can use. But, we need to go for the concepts like Data Providers and JXL (Read data from Excel sheet) while using JUnit or TestNG.

When we use FitNesse, we can simply achieve the above things without Data Provider and JXL. Also, we doesn't require any technical person to execute our scripts. Once the test scripts are configured with FitNesse, any one can execute the scripts from there.

Just to give a try, follow these steps:

1. Download FitNesse from here.
2. Once its downloaded, create a folder called FitNesse in C or D drive and move the downloaded file(fitnesse.jar) into this folder. 
3. Go to command prompt and open this folder C:\FitNesse
4. Type java -jar fitnesse.jar and press enter. FitNesse jar file will be unpacked and you will find a additional folder called FitNesseRoot which will be having all the supporting files
5. Again type java -jar fitnesse.jar and press enter. This time your FitNesse server will be started. If it ask you to choose the different port, use -p argument with the above command: java -jar fitnesse.jar -p 8081
6. Now open your browser and go to http://localhost. You will see the FitNesse environment.
7. You will see some introduction notes on the home page. Click on Edit from left navigation and remove all the notes and configure your Java test scripts and jar files. You need to include the below things: 
• !contents
• !path C:\Fitnesse\FitNesseRoot\files\classes
• !path C:\Fitnesse\lib\*.jar
8. Make sure you have created the folder called "classes" under C:\Fitnesse\FitNesseRoot\files location and included all your test scripts (Java class files). Also make sure you have created lib folder under C:\FitNesse and included all the necessary jar files: FitNesse.jar, selenium-java-client-driver-2.x.jar, selenium-server-standalone.jar
9. Once you have configured your test scripts and jar files, click on save to update the changes. Now you can only see your test script and jar file configuration paths on your FitNesse home page. Whenever you open FitNesse, you will see this path.
10. Now click on Add Child on the top of the home page to create a Test Suite and Test and enter a name. Make sure you have minimum two words in the name and each word should start with capital letter. For Example, TestSuite - SampleSuite and Test - SearchCheck.
11. Now you can see your suite on the home page. Click on that, you will find your Test. When you open that, you will get Edit option in the left navigation. Click on that and call your test script. Follow the below format: 
• !|package name.class name|
• |parameter name|method name|
• |parameter value|Expected Outcome|
12. For Example, if you have created a class SampleClass under the package Sample in Java. Assume that you have a test method "firstMethod()" and a parameter "searchString"
• !|Sample.SampleClass|
• |searchString|firstMethod()?|
• |Sachin|True|
13. Remember that we are passing the value for the variable searchString from FitNesse, but the variable should be declared in your java code. 
14. If you want to pass number of parameters, you need to include as follows:
• !|Sample.SampleClass|
• |searchString|firstMethod()?|
• |Sachin|True|
• |Yuvraj|True|
• |Dravid|True|
• |Dhoni|True|
15. After adding all your parameters, save the changes and click on Test on the left navigation. But, make sure you test script is created and the class file has been stored in this location: C:\Fitnesse\FitNesseRoot\files\classes
16. I have created a test script with Same class and test method. You can try this example: 

package Sample;

import com.thoughtworks.selenium.DefaultSelenium;

import fit.ColumnFixture;

public class SampleClass extends ColumnFixture {

public DefaultSelenium selenium;

public String searchString;

public SampleClass()

{

selenium = new  DefaultSelenium("localhost", 4444, "*firefox", "http://www.google.com/");

selenium.start();

}

public boolean firstMethod() throws Exception

{

selenium.open("/");

selenium.waitForPageToLoad("30000");

selenium.type("name=q", searchString);

selenium.click("btnG");

Thread.sleep(5000);

return selenium.isTextPresent("News for "+searchString);

}

}

Screenshot of FitNesse test environment:

JUnit or TestNG annotations gives syntax error

Sometimes JUnit or TestNG annotations gives syntax error eventhough we have imported all the necessary packages. It gives the error "Syntax error, annotations are only available if source level is 5.0". When you hover at the annotations, you will be able to see this error.

Actually nothing wrong with your JUnit or TestNG packages/jars. This is something to do with your Eclipse setup. This will happen if you have JDK 1.3 or 1.4 installed or configured with your Eclipse. If you have JDK 1.5 or 1.6 installed/configured you can fix this issue. Make sure which version of JDK has been installed/configured.

Follow these steps to get this issue fixed.

1) Eclipse > Window Menu > Preferences - Expand Java - Select Compiler - Check the version of JDK Compilance level.
2) If you see JDK 1.3 or 1.4, select JDK 1.5. or 1.6 for all the options and click on Apply.

Browser window not getting maximized while using Selenium windowMaximize() method

Unlike QTP, selenium tests can be executed without maximizing the browser window. Unless you click on the browser window, it will not get maximized. So that we can work on other things when the tests gets executed. Though its useful for us, sometimes we want to see whats going on in the web application and we maximize the browser window from taskbar.

Even we have Selenium windowMaximize() method to maximize the web application. But, if you see, it does maximize!!!, actually it does, but the application (broswer window) will not get the focus. When selenium.windowMaximize() gets executed, the application gets maximized and still it stays in the task bar. Manually we need to click on the browser tab on the taskbar to get the focus of that application.

Normally we use the below code for starting Selenium and maximizing the browser window.

@Before
public void setUp() 
{
         selenium = new DefaultSelenium("localhost", 4444, "*firefox", "http://www.google.co.in/");
         selenium.start();
         selenium.windowMaximize();
         selenium.windowFocus();
}

@Test
public void testUntitled()
{
         selenium.open("/");
         .............................
         .............................
}

But, most of the times your browser window doesn't get maximized. Some time it gets maximized only if you dont have any other window opened in your taskbar. We have a workaround to resolve this issue. Thats Java Robot class.

We can use Java Robot class to minimize all the windows before opening the URL using Selenium object. We need to activate Window and M keys using robot class. This will minimize all the applications opened. Then we can call selenium.windowMaximize() method which will maximize your browser window. Use the below code.


@Before
public void setUp() 
{

         selenium = new DefaultSelenium("localhost", 4444, "*firefox", "http://www.google.co.in/");
         selenium.start();
         Robot robot = new Robot();
         robot.keyPress(KeyEvent.VK_WINDOWS);
         robot.keyPress(KeyEvent.VK_M);
         robot.keyRelease(KeyEvent.VK_WINDOWS);
         robot.keyRelease(KeyEvent.VK_M);
         Thread.sleep(1000);
         selenium.windowMaximize();
         selenium.windowFocus();
         selenium.windowMaximize();

         selenium.windowFocus();
}

@Test
public void testUntitled()
{
         selenium.open("/");
         .............................
         .............................
}

Hope this code helps to maximize your browser window when you execute your Selenium tests.

Handling Certificate errors on Firefox and IE using Selenium

When we test some sites, we might get SSL Certificate errors and it will be annoying for the Automation testers. They need to manually click on Add Exception which doesn't help when we go for executing Automation regression packs. Its not possible for the tester to click on Add Exception each and every time the critificate error pops up. The reason for getting this certicate error is that RC creates new Firefox profile every time the scripts gets executed.

We have some workarounds for handling these errors in Firefox and IE.

Firefox:

For Firefox, we have two workarounds.

First Workaround:

When you start your Selenium server, you need to add one parameter -trustAllSSLCertificates which will make your Selenium server as proxy for Firefox browser session. Then, it can easily handle any HTTPS website.


java -jar selenium-server-standalone-2.2.0.jar -trustAllSSLCertificates

Second Workaround:

We should not allow Selenium RC to create a new Firefox profile every time. Instead we can create a dedicated profile and ask Selenium RC to use that every time. Follow the below steps:

1) Close all Firefox windows opened.
2) Go to cmd prompt and C:\Program Files\Mozilla Firefox>
3) From Mozilla folder, open Firefox Profile manager using this command, firefox -ProfileManager

4) Create a new profile for Selenium in ProfileManager dialog box and select the folder C:\Profile.
5) Choose that profile and click on Start Firefox.
6) Once Firefox loads, open the page which gives certificate error and add the exception manually.
7) Close your firefox session.
8) Now start the selenium server with the following command:
java -jar selenium-server-standalone-2.2.0.jar -firefoxProfileTemplate C:\Profile

The server will be started with the Firefox profile which you have created. When we execute our scripts, server will launch that Firefox profile.

Internet Explorer:

For Internet Explorer browser, it will be simple and easy. We need to add a small code just after selenium.open() statement.

Ex:

@Test
 public void testMethod()
{
        selenium.open(url);
        if (selenium.isElementPresent("overridelink"))
        {
            selenium.click("overridelink");
            Thread.sleep(30000);
        }

        ....................................
        ...................................
}

Handling Authentication dialog box using Selenium and AutoIT

As we know Selenium is an automation tool for web based applications and it handles only web elements and popups which triggered from web pages. Sometimes we get Window based popups like Authentication dialog box, Security warnings while using the web pages. The question would be how to handle these popups from Selenium?

The answer would be there is no straight forward method or command in selenium to handle these popups. We can use getAlert() and ChooseOKOnNextConfirmation() methods only when the popups are triggered from web pages.

In this scenario, we need to go for AutoIT scripts which can handle these kind of popups. These scripts can be integrated with Selenium code. Follow these steps to handle Authentication dialog box using Selenium and AutoIT.

1. Download AutoIT tool. It can be downloaded from http://www.autoitscript.com/site/autoit/downloads/

Note: Please download AutoIT Full Installation

2. While installing AutoIT, you will get two options: Run the Script and Edit the Script. If you are going to use the script without any modification, choose Run the Script option. If you want to customize the scripts, choose Edit the Script option (Recommended as we always needs to customize the scripts for our needs. Sometimes we need to create our own scripts.)

3. Once installed you can find all the sample scripts in the Example folder. Assume we have a sample script for handling the Authentication box which is Login.au3 (extension of the AutoIT scripts will be .au3). Right click on that file and choose Edit option. Find the script below:

WinWaitActive("Authentication Required","","20")
 If WinExists("Authentication Required") Then
 Send("userid{TAB}")
 Send("password{Enter}")
 EndIf

4.Customize the scripts according to your need. 

For example, You can type your username and password in the script.

WinWaitActive("Authentication Required","","20")
 If WinExists("Authentication Required") Then
 Send("selenium_user{TAB}")
 Send("selenium123{Enter}")
 EndIf

5. Save the file once you modify all the required changes.

6. Now, open the application Aut2Exe which will convert your AutoIT script into EXE format.

7. Now, call this EXE file from your selenium code as below.

@Test
public void testUntitled()
{
         Runtime run = Runtime.getRuntime();
         Process pp=run.exec("C:\\Program Files\\AutoIt3\\Examples\\Login.exe");
         selenium.open(URL);
         ...................
         ...................
}

8. Execute you selenium code and check if the AutoIT script handles the Authentication Required dialog box.

Even we can handle File Save As, Open With dialog boxes using AutoIT and Selenium. It will be covered in my coming blogs. Stay connected :-)