Parsing Text Files in Java

The following code is designed to parse (comma, tab, etc.) delimited files in Java.

private static ArrayList parseDelimitedFile(String filePath, String delimiter) throws Exception
{
  ArrayList rows = new ArrayList();

  FileReader fr = new FileReader(filePath);
  BufferedReader br = new BufferedReader(fr);

  String currentRecord;
  while((currentRecord = br.readLine()) != null)
    rows.add(currentRecord.split(delimiter));

  br.close();

  return rows;
}

How to Concatenate Two String Values in SQL

(The following example was tested in SQLite, but it should work for most versions of SQL.)

The syntax for combining two string values is the following.

SELECT Column1 || Column2
FROM TheTable

How to Do an IF Statement in SQL

(The following example has been tested in SQLite, but it should work for most versions of SQL.)

Technically there is not a way to do IF statements in the SQLite query language like there is in some other versions of SQL. But you can essentially do it with the CASE statement. The following statement is similar to doing an IF, ELSE IF, ELSE statement.

SELECT (CASE InRome
WHEN ‘Yes’ Then ‘Do As the Romans Do’
WHEN ‘No’ Then ‘Do As the French Do’
ELSE ‘Who knows what to do’
)

Find Files in Directory Using Python

A nice solution to this is the path Python module. However, the following simple solution will do the trick. It doesn’t support wildcards at this point, but that could easily be added with some regular expression code.

def getFilesMatchingPattern(directory, nonWildCardPattern):
  fileList=os.listdir(directory)
  return [f for f in fileList if f.find(nonWildCardPattern) > -1]

Converting a String to a Boolean in Python

Let’s say you have a string value that you want to convert to a boolean, but you’re not sure the format it will be in. Some languages have built-in functions for doing this, but to my knowledge Python doesn’t. Here’s a way to do it (though it’s not comprehensive). (Thanks to the commenter who helped me see a simpler way to do this.)

def parseBoolString(theString):
  return theString[0].upper()==’T’

parseBoolString(“true”)

True

parseBoolString(“false”)

False

Sorting Dictionaries in Python

Newer versions of Python have a built-in function called sorted that can help you sort dictionaries. Below is the basic functionality.

Sort by key:

sorted(x.items())

Sort by value:

class SortedDictionary:
  def __init__(self, dictToSort):
    self.keys = sorted(dictToSort.iterkeys())
    self.values = [dictToSort[key] for key in self.keys]
    self._lastIndex = -1

  def __iter__(self):
    return self

  def next(self):
    if self._lastIndex < (len(self.keys) - 1):
      self._lastIndex += 1
      return (self.keys[self._lastIndex], self.values[self._lastIndex])
    else:
      raise StopIteration

x = {}
x['abc'] = 1
x['aaa'] = 2

y = SortedDictionary(x)
print y.keys
print y.values

for z in y:
  print z

Copying from One Directory to Another in Linux

cp FileName.txt DirectoryName/

Append a List to a List in Python

(Note: Please see my latest posts at my new blog!)

An easy way to do this is with the extend function:

x = [1,2,3]

x.extend([4,5])

[1,2,3,4,5]

Simple Method to Calculate Median in Python

(Note: Please see my latest posts at my new blog!)

def getMedian(numericValues):
  theValues = sorted(numericValues)

  if len(theValues) % 2 == 1:
    return theValues[(len(theValues)+1)/2-1]
  else:
    lower = theValues[len(theValues)/2-1]
    upper = theValues[len(theValues)/2]

    return (float(lower + upper)) / 2  

def validate(valueShouldBe, valueIs):
  print “Value Should Be: %.6f, Value Is: %.6f, Correct: %s” % (valueShouldBe, valueIs, valueShouldBe==valueIs)  

validate(2.5, getMedian([0,1,2,3,4,5]))
validate(2, getMedian([0,1,2,3,4]))
validate(2, getMedian([3,1,2]))
validate(3, getMedian([3,2,3]))
validate(1.234, getMedian([1.234, 3.678, -2.467]))
validate(1.345, getMedian([1.234, 3.678, 1.456, -2.467]))

Filtering Data in Python (Example of Functional Programming Approach)

(Note: Please see my latest posts at my new blog!)

Let’s say you are doing a cancer study and have a list of patients of various ages in a tab-delimited file. You want to limit the study to patients who are 60 years or older. One way you could do this is use a for loop and process the data one row at a time and remove any patients below your threshold. Another way is to insert the data into a SQL database and use a WHERE clause to filter the data and then extract it back out. (That’s pretty desperate, but I know it happens!)

One simple way to do this in Python is to use the filter function. Let’s say you pull the data from the file into a series of tuples.

file = open("Patients.csv", 'r')
patients = [line.rstrip().split('\t') for line in file]

Now suppose age is the 3rd column in the data. You need to create a small function to determine whether a tuple meets the criteria:

def f(x): return int(x[2]) >= 60

Then you use the filter function and apply that function to the data.

matches = filter(f, patients)

This is a contrived example, but I hope it illustrates a beginning of how you might use functional programming and that it gives you a flavor for how this can be a powerful approach.