back

by freakynit·1y ago·view on hn ↗
Hey, thanks...

Almost all the major work is done by sqlite (packaged as wasm binary), with second most by Papaparse library. I just joined these two and put up a decent web ui on top. The only manual thing done by me is schema parsing, which is based on first 1000 rows, and downloading functionality.

1 comments
How are you doing the schema parsing?
(1.) Since this is a javascript environment, the types are pretty minimal and just one of each top level type (unlike languages like java or C, which, for example, clould have int, float, double, etc. for just one `number` top-level type). These, here, in javascript, are: number, string, boolean, object, null/undefined.

(2.) Then I use `typeof` operator for each field to determine appropriate javascript type.

(3.) Then, I map these javascript types to sqlite types using this mapping:::

const typeMapping = { 'number': 'REAL', 'string': 'TEXT', 'boolean': 'INTEGER', 'object': 'TEXT', 'undefined': 'TEXT' };

(4.) Then, I generate create table sql's fields declaration part. This also handles an edge case where if first 1000 rows (used for schema generation) of a particular field are all null/undefined, I default to using sqlite's TEXT type using this:::

const columns = fields.map(field => `'${field}' ${typeMapping[columnTypes[field]] || 'TEXT'}`);

(5.) Finally, I create sqlite table:::

const createTableSQL = `CREATE TABLE data (${columns.join(', ')});`;

db = new SQL.Database();

db.run(createTableSQL);

Thanks..

Nice explainer, thanks!
Welcome :)