README.md 1.41 KB
Newer Older
1
# Metagrid Data
tobinski's avatar
tobinski committed
2
This repository collects and updates csv daa imported into metagrid. We usually try to crawl data from existing websites or APIs, but some providers just offer csv dumps. They can't provide a valid sitemap, do not have a website or have no structured data.
3
4
## Provider
- parlament.ch (rm-1848-inkl-id-1.csv) Members of the swiss parlament since 1848
tobinski's avatar
tobinski committed
5
6
7
- ~~SSRQ (SSRQ_pers_26_11_2015.csv). Members of the DB from the "Sammlung Schweizerischer Rechtsquellen"~~ We crawl them directly
- ~~BSG (bsg.csv). All members of the "Bibliohgraphie zur Schweizergeschichte"~~ We use the oapi interface
- ~~Helveticat (helveticat.csv). All authors from the catalog of the "Schweizerischen Nationalbibliothek (NB)"~~ We use the oapi interface
Tobias Steiner's avatar
Tobias Steiner committed
8
9
- Eth Hochschularchiv (2019-02_Hochschularchiv_ETH_Zürich_GND_VEs_puliziert.csv). All proffessors of the eth
- Gechischte der Sozialen Sicherheit (geschichtedersozialensicherheit.csv). The csv is generated with a spider. See the corresponding project
Tobias Steiner's avatar
Tobias Steiner committed
10

tobinski's avatar
tobinski committed
11
# Metagrid v2
12
13
14
15
16
## Import
For each provider there is a special importer in metagrid. Importers handle the mapping between the csv and metagrid's datamodel. To start an import use the following command:
```bash
php bin/console MetagridApiBundle:import:csv <path to csv> <slug of the provider>
```
tobinski's avatar
tobinski committed
17
18
19
20
21
22
# Metagrid v3
## Import
Each provider fetches the csv from this repo and imports it into the datastore. 
```bash
metagrid collect <slug-provider>
```
Tmen's avatar
Tmen committed
23