If you don’t have FME running already, get started with a trial straight from the Google Cloud Marketplace, and check out these instructions on how to deploy FME in Google Cloud. (Note: FME has example files for loading.) The example starts with the assumption that the user has data to load, a shape file for an example, and a licensed or trial version of FME Desktop installed on a virtual machine or on their local machine. I'm excited that my team has such a powerful tool available to them." How to load geospatial data from FME to BigQueryĬheck out this detailed look at how the loading process works.
The addition of the BQ writer to FME is game-changing for us. “We've imported over 1,000 files to BigQuery using FME in just a few weeks. "We've been using FME to transform shp, GeoJson, CAD and various other data types for several years,” says Adam Radel, IT director for GISP, State of Utah Department of Transportation. We’ve heard from customers using FME and BigQuery that they can move and transform data more quickly to focus on innovation. That way you can make sure the data migrated to data warehouses like Google BigQuery is valid and meets all requirements. The transformers are accessible through the GUI and let you create consistent and repeatable spatial data pipelines. With data coming from different sources, data validation and quality control is a critical step in your workflow. Here’s a look at the FME Workbench interface, the authoring environment for data integration workflows.įME supports hundreds of formats, applications, and systems, and includes 493 transformers to help data manipulation tasks like geometry and geography creation, validation, generalization, and coordinate system reprojections. Workflows are scalable: When you build a data integration workflow in FME, you can ingest a single file or hundreds at a time, transform them, and load them directly into BigQuery tables, all within FME. In this case, BigQuery GIS uses the WGS84 coordinate system. Loading them into a data warehouse requires transforming the data type and its projection into the native projection of the data warehouse. Using FME to connect to BigQuery GISThere are hundreds of GIS file types and projections. Plus, you can build event-based workflows to automate your data integration tasks, create notification services, and take advantage of real-time processing. Using a visual interface, you can build workflows to extract, transform, load, integrate, validate, and share data. FME is designed to help you overcome common data integration challenges. To accelerate the workflows of our geospatial customers, we’re announcing our partnership with Safe Software, the maker of FME.įME is a data integration platform designed to support spatial data worldwide, and 2020.0 brings the ability to ingest data from more than 450 geo formats and applications and materialize them as BigQuery tables.įME is ideal to use when you need to ingest and transform one of the myriad geospatial file and data formats and land that data in BigQuery. With this unique capability, you can process and analyze geospatial data at scale. With all of the geographical data available today, we made sure that Google Cloud’s BigQuery data warehouse includes first-class support for geospatial data types and functions.