0

I'm attempting to read a certain file in Java Whenever I read a line of code from the script, It says:

Welcome to DrJava.  Working directory is /Users/anu-lion
> run WebCrawler02
!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="ena
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
    at java.util.ArrayList.rangeCheck(ArrayList.java:635)
    at java.util.ArrayList.get(ArrayList.java:411)
    at WebCrawler02.main(WebCrawler02.java:147)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at edu.rice.cs.drjava.model.compiler.JavacCompiler.runCommand(JavacCompiler.java:272)
> 

I know that this error is caused when the coding can't reach the specific index, but I have no idea how to fix it at the moment.

Here is my coding .

import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.util.ArrayList;
import javax.swing.JOptionPane;

public class WebCrawler02 {
            ArrayList<String> pagesVisited;
            ArrayList<String> pagesToVisit;

            public static URL getStartingURLFromUser() {
                        // This method gets the URL from user.
                        // Initializes the URL string.
                        URL startingURL = null;
//                     String startingString = "http://www.cs.uwec.edu/~stevende/cs145testpages/default.htm";
                         String startingString = JOptionPane.showInputDialog(null, "Enter a valid URL:");
                        startingString.toLowerCase();
                        // catches errors in URL.
                        try {
                                    startingURL = new URL(startingString);
                        } catch (MalformedURLException e) {
                                    System.out.println("Bad URL");
                                    System.exit(0);
                        }
                        return startingURL;
            }

            public static String htmlReader(URL webURL) {
                        // Initializes the strings.
                        String htmlContent = null;
                        // Reading the file.
                        // This try/catch block reads in a file with a reader and creates a
                        // continuous string.
                        try {
                                    URLConnection con = webURL.openConnection();
                                    InputStream is = con.getInputStream();
                                    BufferedReader br = new BufferedReader(new InputStreamReader(is));
                                    String line = br.readLine();
                                    while (line != null) {
                                                htmlContent = htmlContent + line + " ";
                                                line = br.readLine();
                                    }
                                    br.close();
                        } catch (MalformedURLException e) {
                                    System.out.println("Bad URL");
                                    System.exit(0);
                        } catch (FileNotFoundException e) {
                                    e.printStackTrace();
                                    System.out.println("File not found.");
                        } catch (IOException e) {
                                    e.printStackTrace();
                                    System.out.println("after try/catch");
                        }
                        return htmlContent;
            }

            public static ArrayList<String> linkParser(String htmlContents) {
                        // Initializes the strings, boolean, variables and arraylist.
                        ArrayList<String> listOfLinks = new ArrayList<String>();
                        int i = 0;
                        int k = 0;
                        boolean search = true;
                        String URL = "";
                        // This loop creates an array list of URL's.
                        while (search == true) {
                                    i = htmlContents.indexOf("HREF=", k) + 6;
                                    k = htmlContents.indexOf("\">", i);
                                    URL = htmlContents.substring(i, k);
                                    listOfLinks.add(URL);
                                    if (i == htmlContents.lastIndexOf("HREF=") + 6) {
                                                search = false;
                                    }
                        }
                        return listOfLinks;
                        // System.out.println(htmlContents);
            }
            public static boolean isBrokenLink(URL currentURL, String theHREF) {
                        // Initializes the strings, boolean, variables and arraylist.
                        Boolean isbroken = false;
                        try {
                                    URL baseURL = new URL(currentURL, theHREF);
                                    URLConnection con = currentURL.openConnection();
                                    HttpURLConnection httpProtocol = (HttpURLConnection) con;
                                    httpProtocol.getResponseCode();
                                    int httpPro = httpProtocol.getResponseCode();
                                    if (httpPro != 200) {
                                                isbroken = true;
                                    }
                        } catch (MalformedURLException e) {
                                    isbroken = true;
                                    System.out.println("Bad URL1");
                        } catch (FileNotFoundException e) {
                                    isbroken = true;
                                    e.printStackTrace();
                                    System.out.println("File not found.");
                        } catch (IOException e) {
                                    isbroken = true;
                                    e.printStackTrace();
                                    System.out.println("after try/catch");
                        }
                        return isbroken;
            }

            public static void displayBrokenLinkReport(
                                    ArrayList<ArrayList<String>> brokenLinks) {
                        System.out.println("Broken Link Report: \n");
                        for (int j = 0; j < brokenLinks.size(); j++) {
                                    for (int i = 0; i < brokenLinks.get(j).size(); i++) {
                                                if (j == 0) {
                                                            System.out.println("Page " + brokenLinks.get(j).get(i));
                                                } else {
                                                            System.out.println("Broken link "
                                                                                    + brokenLinks.get(j).get(i));
                                                }
                                    }
                        }
            }

            public static void main(String[] args) {
                        // Initializes the arraylists and the array<arraylist
                        ArrayList<URL> pagesVisited = new ArrayList<URL>();
                        ArrayList<URL> pagesToVisit = new ArrayList<URL>();
                        ArrayList<String> listOfLinks = null;
                        ArrayList<String> badLinks = null;
                        ArrayList<ArrayList<String>> brokenLinks = new ArrayList<ArrayList<String>>();
                        int i = 0;
                        // calls the getStartingURLFromUser method which returns the string.
                        pagesToVisit.add(getStartingURLFromUser());
                        while (!pagesToVisit.isEmpty()) {
                                    URL baseURL = pagesToVisit.get(0);
                                    pagesVisited.add(baseURL);
                                    pagesToVisit.remove(0);
                                    // calls the htmlReader method which returns the string.
                                    String htmlContent = htmlReader(baseURL);
                                    // calls the linkParser method which returns the array list.
                                    listOfLinks = linkParser(htmlContent);
                                    // calls the isBrokenLink method which returns the string.
                                    for (i = 0; i <= (listOfLinks.size()); i++) {
                                                boolean isBL = isBrokenLink(baseURL, listOfLinks.get(i));
                                                System.out.println(listOfLinks.get(i) + "a");
                                                if (isBL) {
                                                            System.out.println(listOfLinks.get(i));
                                                            badLinks.add(listOfLinks.get(i));
                                                } else {
                                                            // catches errors in URL.
                                                            URL nextURL = null;
                                                            try {
                                                                        nextURL = new URL(baseURL, listOfLinks.get(i));
                                                                        if (!pagesVisited.contains(nextURL)) {
                                                                                    pagesToVisit.add(nextURL);
                                                                        }
                                                            } catch (MalformedURLException e) {
                                                                        e.printStackTrace();
                                                                        System.out.println("Bad URL2");
                                                            }
                                                }
                                    }
                                    brokenLinks.add(badLinks);
                        }
                        displayBrokenLinkReport(brokenLinks);
            }
}

Thanks in advance.

4
  • what is your input that leads you to this error? Aug 21, 2014 at 0:48
  • 1
    Problem is obvious: you are trying to pull index 1 out of an ArrayList of length 1 (so the only valid index is 0). You need to do a bit more work to show the area of code that is causing the problem - the exception tells you, but not many people are going to go through all of your code to isolate your problem for you.
    – John3136
    Aug 21, 2014 at 0:49
  • The error's at line 147, wherever that is.
    – Hot Licks
    Aug 21, 2014 at 0:49
  • This is an easy problem to debug. What have you tried??
    – Hot Licks
    Aug 21, 2014 at 0:51

2 Answers 2

3

for (i = 0; i <= (listOfLinks.size()); i++) I guess the problem is here, try using plain < instead of <=. listOfLinks.size() should be the size of your listOfLinks, and since it is indexed beginning with zero, the last element should have an index of listOfLinks.size()-1.

2
  • 2
    Another way to say the same thing: 0 is the index of the first element, 1 is the index of the second element, listOfLinks.size() - 1 is the index of the last element, and listOfLinks.size() is an index after the last element, i.e. out of the bounds of the list (which explains the name of the exception)
    – Volune
    Aug 21, 2014 at 0:58
  • @Volune: +1, that's a good way to phrase this =)
    – user3079266
    Aug 21, 2014 at 1:04
2
for (i = 0; i <= (listOfLinks.size()); i++) {

should be

for (i = 0; i < (listOfLinks.size()); i++) {

on line 145.

Not the answer you're looking for? Browse other questions tagged or ask your own question.