
Warp dsdgen for large scales on a hadoop cluster.

$ ./build.sh

this will compile it and use maven to build the MR app wrapped around it.

To generate the data for scale = 10000

	$ hadoop jar ./target/tpcds-dsdgen-2.6.jar -d /user/beeline/tpcds/10000/ -s 10000
 
This uses the existing parallelism in the driver.c of TPC-DS without modification and uses it to run the command on multiple machines instead of running in local fork mode.

The command generates multiple files for each map task, resulting in each table having its own subdirectory.


