Thanks to visit codestin.com
Credit goes to github.com

Skip to content

The newer version of GeoConverter. Takes rows.tsv from FinBIF data download as an input and returns GeoPackage.

License

Notifications You must be signed in to change notification settings

luomus/geoconverter_2

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GeoConverter 2

GeoConverter 2 is a comprehensive tool for converting geospatial data between different formats:

  • TSV to GeoPackage: Converts TSV files (contained in ZIP archives) into GeoPackage format, specifically tailored for FinBIF data downloads
  • GIS to Table: Converts various GIS formats (GeoJSON, GeoPackage, KML, GML, Shapefiles) into CSV format with WKT geometries

Features

  • Support for multiple input formats: ZIP archives with TSV data, GeoJSON, GeoPackage, KML, GML, Shapefiles
  • Multiple geometry processing options: original footprints, centroids (points), or bounding boxes
  • Coordinate system transformation support (WGS84, EUREF-FIN)
  • Asynchronous processing for large files
  • Docker containerization for easy deployment
  • RESTful API with comprehensive status tracking

API Endpoints

/convert-to-table (POST)

Description: Converts GIS formats (.geojson, .json, .gpkg, .kml, .gml) or compressed GIS files (e.g. .shp inside a .zip) into a CSV file. The filename is inferred from the uploaded file.

Response:

  • Returns converted CSV file, where the geometry is converted to WKT format.

Example Request:

curl -X 'POST' 'http://127.0.0.1:8000/convert-to-table' \
    -H "Accept: text/csv" \
    -H "Content-Type: multipart/form-data" \
    -F "file=@geopackage_in_zip.zip" \
    -o "output.csv"

Note: The Accept header specifies the expected response format, while Content-Type is automatically set by curl when using -F for file uploads but is included here for clarity.

/convert/{lang}/{geo}/{crs} (POST)

Description: Converts a ZIP file containing TSV data into a zipped GeoPackage. The TSV file should have the same schema and content as a downloadable file from FinBIF.

Path parameters:

  • lang: Language of the headers - one of fi, en, or tech
  • geo: Geometry type - one of footprint, bbox, or point
  • crs: Coordinate Reference System (CRS) for output - either wgs84 or euref

Response:

  • Returns a JSON response describing the status of conversion and path to download the output when conversion is ready

Example Request:

curl -X 'POST' 'http://127.0.0.1:8000/convert/HBF.12345/tech/footprint/wgs84' \
    -H "Accept: application/json" \
    -H "Content-Type: multipart/form-data" \
    -F "[email protected]"

/convert/{id}/{lang}/{geo}/{crs} (GET)

Description: Downloads and converts a file by ID from files stored on the FinBIF data warehouse server

Path parameters:

  • id: ID for file download on the FinBIF server
  • lang: Language of the headers - one of fi, en, or tech
  • geo: Geometry type - one of footprint, bbox, or point
  • crs: Coordinate Reference System (CRS) for output - either wgs84 or euref

Query parameters:

  • personToken: User's personToken to check access rights for files with sensitive data

Response:

  • Returns a JSON response describing the status of conversion and path to download the output when conversion is ready

Example Request:

curl -X 'GET' 'http://127.0.0.1:8000/convert/HBF.12345/fi/footprint/wgs84?personToken=your_token_here' \
    -H "Accept: application/json"

/status/{id} (GET)

Description: Gets the status of a file conversion

Path parameters:

  • id: ID for file under conversion

Response:

  • Returns a JSON response describing the status of the given file conversion

Example Request:

curl -X 'GET' 'http://127.0.0.1:8000/status/HBF.12345' \
    -H "Accept: application/json"

/output/{id} (GET)

Description: Downloads the converted file

Path parameters:

  • id: ID for converted file

Query parameters:

  • personToken: Optional, user's personToken to check access right for the file with sensitive data.

Response:

  • Returns a ZIP package containing the converted GPKG file

Example Request:

curl -X 'GET' 'http://127.0.0.1:8000/output/HBF.12345?personToken=your_token_here' \
    -o converted_HBF.12345.zip

/health (GET)

Description: Health check endpoint to verify the service is running

Response:

  • Returns service status

Example Request:

curl http://127.0.0.1:8000/health

Installation and Usage

Docker Deployment (Recommended)

  1. Clone the repository:
git clone https://github.com/luomus/geoconverter_2.git
cd geoconverter_2
  1. Build and run with Docker:

    docker build -t geoconverter_2 .
    docker run -p 8000:8000 geoconverter_2
  2. Test the API:

    curl -X 'POST' 'http://127.0.0.1:8000/convert/fi/footprint/euref' \
        -H "Accept: application/json" \
        -H "Content-Type: multipart/form-data" \
        -F "file=@test_data/HBF.12345.zip"
  3. Check service health:

    curl http://127.0.0.1:8000/health

How It Works

TSV to GeoPackage Conversion

  1. Input Handling: The API accepts ZIP files containing TSV data from FinBIF downloads
  2. Data Processing:
    • Reads TSV files using Dask for efficient parallel processing of large datasets
    • Converts WKT (Well-Known Text) geometries to Shapely objects
    • Supports geometry transformations:
      • Original: Preserves original footprint geometries
      • Point: Converts to centroid points
      • Bbox: Converts to bounding box polygons
    • Handles coordinate system transformations (WGS84 ↔ EUREF-FIN)
    • Processes data in partitions for memory efficiency
  3. Output: Writes processed data to a Zipped GeoPackage format using PyOGRIO. Other files in the zip folder remains the same.

GIS to Table Conversion

  1. Input Handling: Accepts various GIS formats (GeoJSON, GeoPackage, KML, GML, Shapefiles in ZIP)
  2. Processing: Uses GeoPandas to read and process spatial data
  3. Output: Converts geometries to WKT format and exports as CSV

Architecture

  • FastAPI: RESTful API framework with async support
  • Dask: Parallel computing for processing large datasets
  • GeoPandas/Shapely: Geospatial data manipulation
  • PyOGRIO: High-performance I/O for geospatial formats
  • Docker: Containerized deployment

Requirements

See requirements.txt for complete dependency list. Key dependencies include:

  • FastAPI 0.115+
  • Dask 2025.2+ and Dask-GeoPandas 0.5+
  • GeoPandas 1.1+
  • Shapely 2.1+
  • PyOGRIO 0.7+
  • Pandas 2.3+
  • Pydantic Settings 2.9+

License

This project is licensed under the MIT License - see the LICENSE file for details.

About

The newer version of GeoConverter. Takes rows.tsv from FinBIF data download as an input and returns GeoPackage.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages