readme.md 1.69 KB
Newer Older
Tobias Steiner's avatar
Tobias Steiner committed
1
# CSV-readable
Lena Heizmann's avatar
Lena Heizmann committed
2
This library exports an abstract class to simplify the import of data into the geolinker form a csv source. It's very specific for te `geolinker` project.
Tobias Steiner's avatar
Tobias Steiner committed
3

Tobias Steiner's avatar
Tobias Steiner committed
4
## Usage
Lena Heizmann's avatar
Lena Heizmann committed
5
You need to implement three methods of the abstract class. The class helps you to get the data from a csv and abstract the publishing to kafka. You just need to implement a mapping of the csv-schema to the `geolinker` and `linker` kafka topic. 
tobinski's avatar
tobinski committed
6 7 8 9 10 11 12 13 14 15 16 17 18
```typescript
    // you get a array of records from the csv and have to return a concordance
    public abstract transformLinks(records): InterfaceConcordance | null;
    // you get a array of records from the csv and have to return a geolinker
    public abstract transformData(records): InterfaceGeolinker | null;
    // define a key for kafka
    public abstract transformDataKey(records): string | number;
    // optional hook to morph the data before we transform them
    public beforeTransform(record, callback) {
        callback(record);
    }
```
## Example
Tobias Steiner's avatar
Tobias Steiner committed
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
```typescript
import {CsvReadable} from 'src/index'
class Test extends CsvReadable {
    constructor() {
        super();
    }
    
  transformLinks(records) {
    return {
      from: records[1],
      to: [records[2], records[10]]
    }
  }
  
  transformData(records) {
    return {
            id: records[0],
            provider: records[2],
            url: records[5],
            name: records[6],
            alternative_name: records[7],
            typology: records[20],
            country: records[2],
            location: `${records[3]},${records[3]}`,
            modification_date: new Date(),
    }
  }
}

new Test();
Tobias Steiner's avatar
Tobias Steiner committed
49
```
tobinski's avatar
tobinski committed
50

Lena Heizmann's avatar
Lena Heizmann committed
51
We use the class in the [fundmuenzen](https://source.dodis.ch/histhub/fundmuenzen) project.