Practical Four - CSV to HTML Converter

Introduction

COVID-19 has changed our world, and as part of that change, Computer Science and various applications have shined. During isolation, everyone found the value of telecommuting, a concept which CS has used for industry for a while. At CSU, research around COVID-19 has continued throughout the entire pandemic, including the Computer Science department, who has research both in looking at the health data, and the Cyber Security Center is looking at security around data integrity and news. When the pandemic first started, numerous models came out, and the amount of data being collected and shared for all to research. John Hopkins, New York Times, CDC, and The World Health Organization are just a few of many examples of data visualizations of the pandemic.

While building websites like the New York Times is complicated, what isn’t complicated is reading CSV (commas separated value) file, and converting it / outputting an HTML file representing that data. For this assignment, you will write the basics of a conversion program. This is also the first assignment where every file you write will be started from scratch. However, you will still be working with provided code integrating your code with code that is provided.

What you will learn

• loops
• working with arrays
• program arguments

Provided Code

As with the previous two practical assignments, you will want to go through canvas to download the code, and add the files to the project. However, the one difference for this assignment, is that you will be adding some files from scratch. Overall, we are providing the “data layer” for the assignment, and you are building the HTML generator or view layer of the assignment.

Files we provide

We are providing the following files for your project. Let’s look at what they do.

• Logger.java
  Application loggers are a major part of the industry to try to track down errors in code, to keep track of what is going on and more. Simple loggers just write out to the console. Logger.java does just that, if logging (Debug or verbose) is enabled. If they are not enabled, any call to logger doesn’t print! As such, it provides a way to debug your code, and leave the debug lines in your code without ruining the final output of your program.
• CSVReader.java
  This is similar to one your wrote for Practical 3, but is more robust in how it handles the CSV format. You should look through this one, as you may be able to replicate it for Practical 5. You will not directly access CSVReader.java in your code for this Practical.
• Document.java
  Document is a wrapper for CSVReader (creates a layer, encapsulating the data), that reads in the CSV data, and stores it into arrays. You will notice the file is immutable, and simply is a storage container for the CSV data.

Pro Tip: Encapsulation
Encapsulation is a fundamental part of programming. How can you build a layer that hides details of implementation, focusing on providing functionality - that is encapsulation. By creating the document class, your code doesn’t care if we are reading CSV files or JSON file or something else - it just cares that data is stored in Document. We then leave it up to whoever writes Document to figure out where the data is coming from and file formats to look at. Programming involves separating out your ideas.

Downloading Files

You can download the files from zybooks, and add them to a new project - similar to what you did for Practical 2 and Practical 3. For most setups, that means your java files are stored in the src directory. You will also notice a number of CSV files. There are options for you to test with, and eventually build HTML files from.

Take the Quiz!

At this point, you should take the Code Tracing Quiz in canvas.

HTML??

You will also have some questions about HTML to make your life easier for coding. If you haven’t learned HTML, you can go to W3Schools in particular, you will need to learn about the table tag. To save time, don’t learn about all of the HTML world now! You probably don’t need to spend five minutes on it.

We will be making use of the following tags:

<table>
<caption>Table caption</caption>
<thead>
 <tr>
  <th>Table head</th>
 </tr>
</thead>
<tbody>
 <tr> 
  <td>Table element</td>
 </tr>
</tbody>
</table>

Additionally

<h1></h1>
<h2></h2>
<h3></h3>
<h4></h4>

Files you are writing

• Tag.java
  Mostly static methods that return HTML string representations of information.
• DocumentToHtml.java
  The main ‘worker’ of the program, that takes a Document, and saves out an HTML file based on the data in the Document.
• Main.java
  The main driver of the program. It will process command line arguments and determine the command flow.

You will notice there isn’t a Tests file. That is because with this practical we are going to introduce another way to test, via different program entry points.

Step 0: Tag.java - protected static String entity(String str, String tag)

The first thing you want to do is create a Tag.java file. This file will consist of multiple static methods are designed to simply build HTML “tags” as Strings to be used in other programs.

When we think about all html tags, they share the following format in common

<TAGNAME>text</TAGNAME>

The tag name can change a lot, but since this format is shared, we will focus on the principles of abstraction and DRY (Don’t Repeat Yourself), to write a helper method. This method, entity, is a protected method for grading purposes. it will take in two parameters, the text that goes in the tag (str), and the html tag name (tag). You will then use String.format to return a String that matches the format above!

Adding Debugging Lines

This is a great place to work with Logger, and add a debug line in the code.

1
Logger.debugFour("Creating Tag: %s with value %s", tag, str);

What this means is the line will only print with ‘level 4’ debugging is enabled, aka a highly detailed log.

Run Tests

Now, lets test the creation of the entity method. In order to test, you will create a public static void main(String[] args) in Tag.java. After you write it in, you will notice a green arrow (if using IntelliJ) right next to the method name. If you click on it, it will compile and run the Tag class only.

Now, let’s add some tests to your code. Here are a few examples to write in the main.

1
2
System.out.println(entity("Header 1", "h1"));
System.out.println(entity("Header 2", "h2"));

Now let’s try some with the Debug!

1
2
3
4
5
Logger.enableDebug(Logger.DebugLevel.LEVEL4);  
// You can also say Logger.enableDebug(4), I just find the other easier to read.
System.out.println(entity("Table Col", "td"));
System.out.println(entity("Table Header", "th"));
System.out.println(entity("Table Row", "tr"));

Try adding your own tests to make sure the tag is working correctly. Note that we don’t care if it is a valid html tag, just that it has the proper open and closing format.

Pro Tip: HTML
HTML is a markup language, a way of giving format information about documents. However, there is more to html tags. They can also contain attributes. For example, `<H1 id="main_title" class="large">Title</h1>. What that is saying is that it is a header element of top priority, with the identifier of main_title and uses the style class of large. You can then use javascript and cascading style sheets (css) to change the looks of the tag! We won’t worry about allowing attributes for this assignment.

Step 1: Tag.java - HTML H1-H4, TD, TR, TH

Now that you have entity working, let’s create a number of methods that can use it!. All these methods simply call entity. For example:

1
2
3
public static String h1(String str) {
  return entity(str, "h1");
}

The methods you need to create are:

• H1
• H2
• H3
• H4
• TD
• TR
• TH

For each, they are all like the example with h1 above, and with lowercase letters for all the method names like the example.!

Run Tests

For each of the tags, create a test in the main method in Tag.java. While it may seem redundant we are testing every tag, so you will want to make sure each tag works.

Step 2: Tag.java - HTML table, html opening and closing

Now with that said, there are some HTML tags that may not make sense to be stored as a giant String. They are better off written separately. For the tags, html, head, body, and table you will create the following methods:

• public static String tableOpen()
  returns the String “<table>”
• public static String tableClose()
  returns the String “</table>”
• public static String htmlOpening()
  returns the String “<html><head></head><body>”
• public static String htmlClosing()
  returns the String “</body></html>”

Run Tests

You should run some tests now combining them. If you do it right, you can build a small HTML document on your screen with a simple table! Maybe a 3x3 table with random values in it? Try it!

At this point if you want to submit for grading, you will need to create DocumentToHtml.java and Main.java. You do not have to fill out the details, but the autograder requires them to be there. For DocumentToHtml.java it can be an empty class, for Main.java it should contain a main method, but that can also be empty.
Given the above code is worth half your grade, we would recommend submitting now!

Step 3: DocumentToHtml.java - public DocumentToHtml(Document document)

After you have Tag.java working, you will start on the second of three classes you are building for this project. This class is going to be an object that the Main class will eventually build. As such, you are going to need a constructor.

After you create your new class, add a constructor the class, similar to what is show above. The constructor will take in a Document object, and store it so private instance variable.

Run Tests

To create a test for this initial constructor, you will need to instantiate the class.

1
2
Document doc =  new Document("data/BeePivot.csv", "Bee Table");
DocumentToHtml d2h = new DocumentToHtml(doc); 

The above code was the code we used. Note, if you did not store your CSV files in a data directory, your line of code may look slightly different! While this builds the object, there isn’t much you can do with it right now. Let’s fix that!

Step 4: DocumentToHtml.java - protected static String toHtmlRow(String[] row)

For the next two steps, we are going to be working on helper methods. These methods would usually be private, but for grading purposes we need them to be at least protected or internal in scope.

toHtmlRow will take an array, and build an html table row from it. For example,

1
d2h.toHtmlRow(["Captain", "Widow", "Iron", "Wanda", "Vision", "Ant"]);

Would return

<tr><td>Captain</td><td>Widow</td><td>Iron</td><td>Wanda</td><td>Vision</td><td>Ant</td></tr>

Note: capitalization doesn’t matter on the Tags. TR or tr is just fine.

In order to do this, you are going to need to start with an empty String and then loop through the array. Each element in the array, adds/concatenates another TD (column) element to the String. Make use of your Tag.java! Then after ou build the string, call Tag.tr(…) on the String you just built and return the result.

Also, above your loop in this method, you should add the following debug statement.

1
Logger.debug("Writing out row %s", Arrays.toString(row));

Run Tests

This one you can run a bunch of tests on. If the array is empty, you should just return the empty String.

You will also want to add the following call in your main “testing” method.

1
Logger.enableDebug();

and see how that changes the output! It will also be a tool that will help you later.

Step 5: DocumentToHtml.java - protected static String toHtmlTableHeader(String[] headers)

For this step, you are repeating the method you wrote above but using th instead of td. Also, for your debug line, it should look like the following:

1
Logger.debug("Writing out headers %s", Arrays.toString(headers));

Run Tests

Run more tests, make sure it is showing <tr><th>…</th></tr> style formatting.

Step 6: DocumentToHtml.java - public void convert(String outputFile)

Until this point, we have been writing methods that will help us build a HTML table, but we haven’t actually written anything out to an HTML file. Convert does exactly that! It is heart of this program, and takes the data stored in Document, and uses that to write out an HTML file!

In this method, you will need to do multiple things, but the primary part is writing out a file. The parameter passed in is the file you want to write out.

As such, we would recommend writing this method in two parts. The first setup, your try/catch statements, and simply write out to a file. Is it saving to a file? Great, now work on the following.

You will now use all more of the Tags. However, to help you out, here is a skeleton of this method with the tags that you need to call in order.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
try {
    Logger.logDetails("Writing table to %s...%n", outputFile);
    
    // TODO: setup your PrintWriter   

    output.println(Tag.htmlOpening());
    
    // TODO: print the title from the document using the h1 tag! 

    // as  long as there is data,  build the table (hint see null check below)
    output.println(Tag.tableOpen());
    
    // TODO: write the code to print the table headers! 
    // TODO: write code to loop through each 'row' of the doc.getData(), and  print the  row
    // hint: we used a for:each loop,  but any for loop works

    output.println(Tag.tableClose());
    output.println(Tag.htmlClosing());
    output.close();
    if(doc.getData() != null) {
      Logger.logDetails("Table Converted. Total Rows Processed: %d%n", doc.getData().length);
    }

} catch (FileNotFoundException e) {
    e.printStackTrace();
}

You will want to include the Logger calls exactly as they are shown (feel free to add others, or add debug commands). With one exception. We called our class instance variable “doc”, you may have used a different name. If that causes an error, you may want to check names.

Run Tests

For running tests, you will want to run multiple files and make sure they all work. You can open up the HTML files either in intellJ or your web browser (do both), to make sure they are writing out correctly.

Here is one sample test we created:

1
2
3
Document doc =  new Document("data/BeePivot.csv", "Bee Table");
DocumentToHtml d2h = new DocumentToHtml(doc);
d2h.convert("output/test.html");

At this point, you should also turn in your assignment again. We are at the 85/100 point mark! It is worth making sure it is working, before moving onto Main.java.

Step 7: Main.java - protected static int getDebugLevel(String arg)

You have some functioning classes, but you don’t have a fully functional program. For the last part of the assignment, you will be building the main driver class. As you recall when learning some basic linux commands, it is very common for command line programs to have “program arguments”. Your Main.java class will handle the command line arguments.

The arguments require both input file and output file, and they have the following options.

--title=value        Takes the value and makes an HTML H1 above the table.
-v || --verbose      Enable feedback while running
--debug=n || --debug Enable debugging print statements at various debug levels

To handle the debug option, we developed a helper method called getDebugLevel. While this would usually be private, you will need to make it protected for grading purposes. The goal of this method is to parse the debug String. The String has two options

• –debug
• –debug=N
  where N is any whole number.

If –debug is passed into the method, 1 is returned. If –debug=N is passed into the method the integer value of N is returned. If the value of N isn’t a number, return 0. A couple methods to help:

• String .split or String .substring and String .indexOf
• String .contains or String .indexOf
• Integer.parseInt

Note, case should not matter.

Run Tests

Try putting in different variations of –debug, –debug=N, and seeing what they return. The unique thing about putting test statements in public static void main(String[] args) for this class, is that you will have to delete the tests when you need to edit that method.

Step 8: Main.java - protected static String[] processArgs(String[] args)

The processArgs method is the primary spot in which you will look at the program arguments. It takes in a String[] array, and will look at each value in the array in order (hint: use a for each loop). You will then return a String array that contains the following

[Input File Name], [Output File Name], [Title of The Table]

As you look at the various arguments, you will evaluate it for each of the following cases.

• -v or –verbose
  If the argument matches either of these cases (ignore case!), you will call Logger.setVerbose(true)
• –debug
  If the argument contains –debug, you will pass the String into getDebugLevel, and use the value returned to call Logger.enableDebug(int) with the int being the value returned by getDebugLevel.
• –title=String
  If the argument contains –title, you will take the value after the equal sign, and place it into the 3rd (index 2) location of your return array.
• Anything else
  • the first String you come across that doesn’t match above, is going to be your input file.
  • the second String you come across that doesn’t match above, is going to be your output file.
  • We suggest checking for the null value in the array locations.

return the array when completed. You can return an array with null values if the arguments are not correct. For example, only an input file is provided. In all cases, you will return a String array with at least 3 spots allocated. For example,

1
String[] rtn = new String[3];

Run Tests

You should try passing in different array combinations to this method, and seeing the return value. For example

1
2
System.out.println(Arrays.toString(processArgs(["-v", "title=My Header", "input.csv", "output.html"])));
System.out.println(Arrays.toString(processArgs(["title=My Header", "input2.csv", "-v",  "output.html"])));

Step 9: Main.java - public static void main(String[] args) (and info box)

Before you start on the main driver method for your program, copy and paste the following code into your Main.java. We are providing this, so you don’t have to worry about spelling and spaces in this one case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
private static void printInfoBox() {
  System.out.println("Missing Arguments");
  System.out.println("Needs at least two arguments to run, an input file and output file.");
  System.out.println("Example: >java Main input.csv output.html");
  System.out.println("Example: >java Main input.csv output.html -title=My\\ Cool\\ File");
  System.out.println();
  System.out.println("Additional options:");
  System.out.println(infoBoxLine("--title=value", "Takes the value and makes an HTML H1 above the table."));
  System.out.println(infoBoxLine("-v || --verbose", "Enable feedback while running"));
  System.out.println(infoBoxLine("--debug=n || --debug","Enable debugging print statements at various debug levels"));
}

private static String infoBoxLine(String command, String info) {
  return String.format("%-20s %s", command, info);
}

Now, let’s get started working on your main method. The main method does the following:

• Processes the command line arguments (String[] args), by calling processArgs.
• Using the values returned from processArgs, you either
  • Call printInfoBox(), and end the program, if inputFile or outputFile is null.
  • Create A Document object, then a DocumentToHtml object, and then call convert on the DocumentToHtml object. Similar to the tests you wrote for convert, but instead they use the values returned from ProcessArgs.

Run Tests

When running tests for your main method, you are actually modifying the program arguments in the run configuration. You should try a few variations to make sure it all works.

Step 10: Turning in and Reflection

You should now turn in your code, and write the reflection in Canvas. What did you learn? What could you improve upon in the future? How did designing a class from scratch go? The next practical will involve you writing all the code from scratch, but you do have all these classes to help you! Are you ready?

Did working with HTML spark interest in designing your own website? As someone with a course in this department, you already have a web server setup. You can learn about this here. Though many students also setup their git web page as part of their git account.

Going Further
You explored two major areas in computer science with this practical: Data Visualization and Human Centered Computing Going beyond this course, we have two concentrations in the department you may want to look at. If you enjoyed the idea of how to display and interact with the data, HCC would be something good to look with a focus on stats. If you really like getting and analyzing the data, Systems is important in how you do that. You can learn about the research labs at CSU, and many undergrads are encouraged to get involved in research.