Parsing CSV in Java; the easy way :)

I work in a project that has tight deadlines and you could never know when something new comes in with a short notice. In such situations, it's really nice to have some really good libraries to do some stuff for you so that you can completely focus on the application logic.

One such task that came my way was to have a loader for objects of light class. A light object is a representation of a real light device that we'd store in our database (It's an IoT domain project :)). So it was to be done in a couple of hours with testing and deployment. The input for the loader was going to be a csv file with more than 5000 records.

The format of the csv file was as follows
X,Y,OBJECTID
where X gives the value of the longitude, Y gives the latitude and OBJECTID gives the id of the light for the provider.

I could have taken the approach of reading the file line by line and then splitting the line using "," and then accessing each value separately and populating the objects. This would definitely work but there was a problem. The format of the file could tomorrow change which would require me to redo things and keep maintaining it every time. Obviously there's no escaping the maintenance but I wanted it to be minimal. I also wanted most of the portions of the code to be reusable just in case some other provider of light comes in with a same use case. Only the format of the input would change and rest of the code would remain the same. Hence I thought that there should be a cleaner way of doing it.

That is when I thought of looking for good libraries for this job. After reading a few answers on stackoverflow, I decided to go with the uniVocity-parsers library for parsing csv.

uniVocity-parsers library cleanly extracts a csv file into Java objects. You can download the library from here and add it to your java project. Let's see how to go about parsing the above mentioned csv into Light objects.

Let the light model look as follows

package example.java.suchitradaemon.model;
public class Light {
private Double X;
private Double Y;
private String OBJECTID;
public Light() {
super();
}
public Light(Double x, Double y, String oBJECTID) {
super();
X = x; Y = y; OBJECTID = oBJECTID; }
public Double getX() {
return X; }
public void setX(Double x) {
X = x; }
public Double getY() {
return Y; }
public void setY(Double y) {
Y = y; }
public String getOBJECTID() {
return OBJECTID; }
public void setOBJECTID(String oBJECTID) {
OBJECTID = oBJECTID; }
}

If you see in the above model, there are three fields X,Yand OBJECTID corresponding to the same fields in the csv. To make use of the library, we'll add some annotations to the above model to let the library know which object fields to be filled in with which fields from the csv.

Following is the modified model class

package example.java.suchitradaemon.model;
import com.univocity.parsers.annotations.Parsed;
public class Light {
@Parsed(field = "X")
private Double X;
@Parsed(field = "Y")
private Double Y;
@Parsed(field = "OBJECTID")
private String OBJECTID;
public Light() {
super();
}
public Light(Double x, Double y, String oBJECTID) {
super();
X = x;
Y = y;  OBJECTID = oBJECTID;
}
public Double getX() {
return X;
}
public void setX(Double x) {
X = x;
}
public Double getY() {
return Y;
}
public void setY(Double y) {
Y = y;
}
public String getOBJECTID() {
return OBJECTID;
}
public void setOBJECTID(String oBJECTID) {
OBJECTID = oBJECTID;
}
}

As you see in the above class, we've added @Parsed annotations to the fields in the class. It lets the library know which field from the csv corresponds to which field in the class. For example, consider the annotation @Parsed(field = "OBJECTID") for the field OBJECTID in the class. It lets the parser library know that the value of the field OBJECTID from the csv should be assigned to the OBJECTID field of the class.

Now let's look at how to actually use the library.  Following is the main class.

package example.java.suchitradaemon;
import java.io.InputStreamReader;
import java.util.List;
import com.univocity.parsers.common.processor.BeanListProcessor;
import com.univocity.parsers.csv.CsvParser;
import com.univocity.parsers.csv.CsvParserSettings;
import example.java.suchitradaemon.model.Light;

public class Loader {
public static List readCSV(String filename, Class beanType) {

BeanListProcessor rowProcessor = new BeanListProcessor(beanType);

CsvParserSettings parserSettings = new CsvParserSettings();

parserSettings.setRowProcessor(rowProcessor);

parserSettings.setHeaderExtractionEnabled(true);

CsvParser parser = new CsvParser(parserSettings);

parser.parse(new InputStreamReader(Loader.class.getResourceAsStream(filename)));

return rowProcessor.getBeans();

}
public static List<Light> readLightInstances(String filename) {
String fname = "/example/java/suchitradaemon/data/ImpStreetLights.csv";
if (null != filename && 0 != filename.length()) {
fname = filename;
}
return readCSV(fname, Light.class);
}
public static void main(String[] args) {
List<Light> lights=readLightInstances(null);
lights.stream().forEach(light->{System.out.println("X: "+light.getX()+"\tY: "+light.getY()+"\tOBJECTID: "+light.getOBJECTID());});
}
}

In the above code, we're calling the function readLightInstances from the main function. It internally calls the readCSV function which is responsible for reading the csv file and returning the java objects corresponding to the csv rows. For each row, the parser creates a new object of the light. The parser returns a list of lights which we can consume to do further operations. In the code above, I am just printing out each light object's field values. 

The readCSV function in the above code is generic and will return the objects of the specified type from the file passed as argument. It takes two arguments; filename and the class which it should look for the parsing information (annotations). We pass it the filename and the Light class and it returns as list of Light objects. It's that hassle free and I could focus on getting my application logic ready with this task off my shoulders.

Hope this helps you in focusing on what matters in your code and leave the parsing to the expert :)

Notes:
1. Note the usage of Loader.class.getResourceAsStream(filename) instead of a FileInputStream and the path of the file being a package path rather than the system path. This is done because I had to export my project as a runnable jar and using system paths do not work with runnable jars. This method of using resources work for java applications as well as for jars and hence the choice :)

2. The line lights.stream().forEach(light->{System.out.println("X: "+light.getX()+"\tY: "+light.getY()+"\tOBJECTID: "+light.getOBJECTID());}); will require JDK 1.8 and above.


In the next installment, I'll show how to do the reverse, Java objects to csv.