Commit 27c294e6 authored by Tobinsk's avatar Tobinsk

Merge branch 'go-metagrid' into 'master'

Go metagrid

See merge request metagrid/metagrid-data!1
parents e369a69d e6e930ba
# Metagrid Data
This repository collects and updates all the data imported into metagrid. We usually try to crawl data from existing websites, but some providers just offer csv dumps. They can't provide a valid sitemap, do not have a website or have no structured data.
This repository collects and updates csv daa imported into metagrid. We usually try to crawl data from existing websites or APIs, but some providers just offer csv dumps. They can't provide a valid sitemap, do not have a website or have no structured data.
## Provider
- parlament.ch (rm-1848-inkl-id-1.csv) Members of the swiss parlament since 1848
- ethz.ch (2017-09_Hochschularchiv_ETH_Z-rich_GND_VEs_puliziert_aufbereittet.csv). Person in the archive of the ethz
- SSRQ (SSRQ_pers_26_11_2015.csv). Members of the DB from the "Sammlung Schweizerischer Rechtsquellen"
- BSG (bsg.csv). All members of the "Bibliohgraphie zur Schweizergeschichte"
- Helveticat (helveticat.csv). All authors from the catalog of the "Schweizerischen Nationalbibliothek (NB)"
- ~~SSRQ (SSRQ_pers_26_11_2015.csv). Members of the DB from the "Sammlung Schweizerischer Rechtsquellen"~~ We crawl them directly
- ~~BSG (bsg.csv). All members of the "Bibliohgraphie zur Schweizergeschichte"~~ We use the oapi interface
- ~~Helveticat (helveticat.csv). All authors from the catalog of the "Schweizerischen Nationalbibliothek (NB)"~~ We use the oapi interface
- Eth Hochschularchiv (2019-02_Hochschularchiv_ETH_Zürich_GND_VEs_puliziert.csv). All proffessors of the eth
- Gechischte der Sozialen Sicherheit (geschichtedersozialensicherheit.csv). The csv is generated with a spider. See the corresponding project
# Metagrid v2
## Import
For each provider there is a special importer in metagrid. Importers handle the mapping between the csv and metagrid's datamodel. To start an import use the following command:
```bash
php bin/console MetagridApiBundle:import:csv <path to csv> <slug of the provider>
```
# Metagrid v3
## Import
Each provider fetches the csv from this repo and imports it into the datastore.
```bash
metagrid collect <slug-provider>
```
This source diff could not be displayed because it is too large. You can view the blob instead.
2018_liens_vers_DHS_VF_20181016.csv
\ No newline at end of file
This diff is collapsed.
2019-02_Hochschularchiv_ETH_Zürich_GND_VEs_puliziert.csv
\ No newline at end of file
geschichtedersozialensicherheit.csv
\ No newline at end of file
This diff is collapsed.
2018-rm-1848-inkl-id-1.csv
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment