The file geodatabase
Presentation overview
Introduction
Comparisons and capabilities
Storage
File system operations, data types, storage limits and
requirements, configuration keywords
Compression Performance vs. other formats
Display, query, load, calculate Performance tips
Migrating to the file geodatabase Additional Information
File geodatabase
How many of you have already seen or used them?
9.2 geodatabase options
Personal GDB File GDB ArcSDE GDB
Personal Workgroup Enterprise
Embeds ArcSDE & database engine
ArcSDE & RDBMS required
Personal GDB
File GDB
ArcSDE Personal
ArcSDE Workgroup
ArcSDE Enterprise
Increasing size and/or functionality
Three geodatabase types
Personal GDB File GDB ArcSDE GDB (3 levels)
Storage format Storage capacity Supported O/S platform Number of users
MS Access 2 GB Windows
Folder of binary files No limits Any platform
RDBMS Depends on server Depends on RDBMS Multiple editors & readers Versioning, replication, archiving
Single editor Multiple readers None (check in / checkout replication only)
Single editor* Multiple readers None (check in / checkout replication only)
Versioning support
Why create a new format?
Users asked for alternative to personal geodatabase Reduce storage requirements Eliminate personal gdb 2 GB limit
Personal gdbs slow after ~ 500 MB
Make available to non-Windows platforms
Eliminate dependency on JET engine
Add ability to lock geodatabase data to ArcReader
Introducing the file geodatabase
New geodatabase format Stores a geodatabase in a folder of files
Like a folder of shapefiles No access to individual datasets via file system
Alternative to Access-based personal gdbs
High Performance Reduced memory requirements Removes database size limits Works on additional operating systems (cross platform)
Similar to other geodatabases
Supports the full Geodatabase model
Features, Annotation, Dimensions, Raster Networks, Topology Terrain, Geocoding, Representations
Work with file gdbs as you would personal gdbs
Designated with a different extension (.gdb vs .mdb) Single editor, no support for versioning New Locking mechanism
Advantages over personal geodatabase
No storage size limit Improved performance Reduced storage requirements Customize storage
Compression of vector data Configuration keywords (similar to ArcSDE)
Additional raster data management functionality More platforms supported
Windows and UNIX (solaris and linux)
Migration from personal to file
Most users will migrate to take advantage of benefits
Personal geodatabases are not going away Only move if it helps Three reasons some may not migrate
Comfort with Personal and have small databases < 500mb Some use Microsoft Access to perform operations Some store mature, historical, or archives in Personal
Editing file geodatabases
Like personal gdbs, Single-user editing
Does not support versioning Access locks at the database level
mydata.mdb and mydata.ldb
Locking
File gdbs have a new locking model Not a database-wide lock
More than one editor at a time, but on different data
Lock and entire feature dataset
All feature classes in the feature dataset are locked
Lock a standalone feature class Lock a standalone table
Stores as a folder of files
Lock file
Other files may also be present, ex. files that start with d if you are editing.
Contents deliberately vague to discourage use of file system
Datafile: consists of at least a .gdbtable and .gdbtablx Attribute index Also present if there are indexes Spatial index Signature file
File geodatabase at 9.2
Introduction
Comparisons and capabilities
Storage
File system operations, data types, storage limits and
requirements, configuration keywords
Compression Performance vs. other formats
Display, query, load, calculate Performance tips
Migrating to the file geodatabase Additional Information
File system operations
Always use ArcGIS tools, not to the file system Possible folder operations (but discouraged)
Copy geodatabase to another location Rename the geodatabase Delete the geodatabase No one else should be connected
Individual file operations
No operation is valid likely Possible data loss or render the data unusable For example, if you move files to another file geodatabase, you
wont be able to access the data
File geodatabase and permissions
In 9.2, there are no file gdb authentication / authorization capabilities You should not set permissions on individual files If you access a file geodatabase on a CD, the data is read-only You can share a file geodatabase folder as read-only
The read-only user will be able to query and display Users with write access can modify the data even when others
are currently reading the data
Permissions example
Reader, looking at the Roads feature class
Writer, starts editing the Roads Writer, adds some new roads while reader queries Writer, saves their work Reader, does not see new roads Reader, does a refresh and then sees new roads
Storage limits
No database size limit Per table limit: 1 TB (default) Per table limit: 256 TB
Available as a configuration keyword Provided for large rasters
Same data in a file geodatabase takes up less disk space than personal gdbs, shapefiles
Amount of reduction varies by dataset Storage on disk generally reduces by 50 to 75%
Storage comparisons
Shapefile US rivers and streams California roads US census block centroids US traffic analysis zones US counties 2.19 GB 1.23 GB 838 MB 249 MB 3.2 MB Personal gdb Exceeds 2 GB limit 684 MB 1.8 GB 295 MB 3.2 MB File gdb 878 MB 329 MB 705 MB 68 MB 1.6 MB 50%
Raster data
Unmanaged rasters stored like a pgdb
C:\ Student C:\ Student
Riley.mdb
Riley_catalog 1 2 3 F:\Images\R01.sid F:\Images\R02.tiff F:\Images\R03.img 1 2 3
Riley.gdb
Riley_catalog F:\Images\R01.sid F:\Images\R02.tiff F:\Images\R03.img
Riley.idb F:\ Images R01.sid, R02.tiff, R03.img F:\ Images
Riley.idb
R01.sid, R02.tiff, R03.img
Managed rasters in a pgdb
Stored as ERDAS Imagine files IDB folder One subfolder per raster Not really inside mdb file However it works like that
ArcCatalog copy, delete, or move
Student Manhattan.mdb
MillerRanch MillerDRG
Manhattan.idb c1 m_1.img c2 m_1.img
Managed rasters in a fgdb
New fgdb Empty
No GIS data
But lots of files
New File Geodatabase.gdb
Add a raster to the empty fgdb
Stored in the gdb folder Really inside gdb folder Hard to tell which files are raster
New File Geodatabase.gdb
erDRG
Configuration keywords
Predetermined keywords stored within the geodatabase, cannot be customized Compared to ArcSDE: very few options
None for specific datasets
Vast majority of users should use DEFAULTS DEFAULTS
1 TB per table UTF8 text attribute storage, optimal for latin alphabets
TEXT_UTF16
Use when lots of text in non-latin alphabet
MAX_FILE_SIZE_4GB, MAX_FILE_SIZE_256TB
File geodatabase at 9.2
Introduction
Comparisons and capabilities
Storage
File system operations, data types, storage limits and
requirements, configuration keywords
Compression Performance vs. other formats
Display, query, load, calculate Performance tips
Migrating to the file geodatabase Additional Information
Compression
Compress
Entire gdb Feature dataset Standalone feature class Table Vector data (raster is usually maximally compressed already)
Advantage: Further reduce storage requirements Lossless compression Based on Smart Data Compression (SDC)
Direct access format No uncompressing required
Compression tools
Compress / Uncompress tools
Right-click context menu commands Geoprocessing tools: Data Management Toolbox > File
Geodatabase toolset
Compression ratios
Feature class compression varies
Minimal amount to ratios exceeding 4:1 Key factor: average number of vertices per feature Attribute fields: text, integer and dates compress better than
floats and doubles
For tables, redundancy is the most important factor
Up to ratios exceeding 4:1 More redundancy, the greater compression
Finds and removes redundancy Repeating values, like run length encoding Store value once and a count of how many times it occurs
Compression comparison
Uncompressed Compressed US census block centroids California roads Calgary buildings US rivers and streams Mexico roads 705 MB 329 MB 48 MB 878 MB 3.5 MB 162 MB 83 MB 20 MB 442 MB 2.7 MB
Ratio 4.4 3.9 2.4 2.0 1.3
Less vertices / feature = more compression
Compression implications on editing
Editing not allowed on a compressed dataset Mixed state - compressed and uncompressed feature classes in one feature dataset
Compress a feature dataset Then make new feature class New feature class is uncompressed But you cant edit it
If a feature dataset or relationship class contains a compressed feature class, participating feature classes cannot be edited
Post-compression
Properties that cannot be modified after compress
Coordinate system information, tolerance Subtypes, domains, default values Fields (add, delete, modify properties) Spatial index Representations
Properties that can be modified after compress
Alias (for feature class / table name) Attribute indexes Metadata
More post-compression rules
Properties of a compressed feature dataset cannot be modified
Coordinate system information Cannot create topology or geometric network from compressed
feature classes Cannot modify relationship class, topology, geometric network, network dataset properties
Properties of a compressed geodatabase can be modified:
Domains
File geodatabase at 9.2
Introduction
Comparisons and capabilities
Storage
File system operations, data types, storage limits and
requirements, configuration keywords
Compression Performance vs. other formats
Display, query, load, calculate Performance tips
Migrating to the file geodatabase Additional Information
Display and Query performance
Compared to shapefiles
Generally comparable Shapefiles store geometries separate from attributes
sometimes faster for non-symbolized drawing
Compared to personal GDB
Faster, both locally and over the network 20% to > 10x faster is common Especially true for personal geodatabases over ~ 500 MB
Uncompressed vs. Compressed
Generally comparable
Load performance
Loading shapefiles into file geodatabases is faster than loading into any other type of geodatabase
1.5 -2 x faster than loading into personal geodatabase 2-2.5 x faster than loading into ArcSDE
Copy / Paste into a file gdb is also faster than into any other gdbs
Performance: tips
Defrag disk occasionally Leave sufficient disk space Spatial index grid sizes
in rare cases may need adjustment
Compact the geodatabase
on a regular basis if you frequently add / delete data after any large-scale change
XY resolution
If data is not as accurate as the default, set a larger resolution
when you create the data
File geodatabase at 9.2
Introduction
Comparisons and capabilities
Storage
File system operations, data types, storage limits and
requirements, configuration keywords
Compression Performance vs. other formats
Display, query, load, calculate Performance tips
Migrating to the file geodatabase Additional Information
Migrating reason review
Reasons to migrate from personal gdbs
No size limit Improved performance Reduce storage UNIX
Reasons not to migrate
Very small datasets only, no advantage to moving Require ability to leverage Access Have mature data already in pgdb
Most users will benefit from migrating
Standard data conversion tools
From a personal geodatabase
Copy/Paste (for feature datasets, classes, and tables) Export to XML Workspace Document (for geodatabases) Existing GP conversion tools
From shapefiles, coverages or other formats
Right-click and Export Existing GP conversion tools
Models or scripts for moving many datasets Creating new datasets
Works the same as for personal geodatabases
SQL statement syntax differences
FGDB SQL similar to shapefile, coverage FGDB SQL differs from personal geodatabase
supports a subset of features and functions syntax differs slightly
Dialogs you create SQL expressions with help you with the correct syntax
appropriate delimiters for fields and values relevant keywords and operators
SQL statements for a personal gdb layer may not work after migration
Definition queries, saved queries, label queries FGDB does not have some functions, Distinct, GroupBy,
OrderBy
Syntax differs from personal geodatabases
Delimit fields with field, not [field] Precede dates with date, not #
[Birth] = #04-11-1963# Birth = date 1963-04-11 PGDB syntax FGDB syntax
String searches case sensitive UPPER and LOWER, not UCASE and LCASE
[Name] = redlands LOWER(Name) = redlands
Wildcards are _ and %, not ? and *
[Name] Like ?edlands Name Like _edlands
Migrating ArcObjects applications
Update WorkspaceFactory to get the app working on file gdb
Change AccessWorkspaceFactory to FileGDBWorkspaceFactory Change extenstion from .mdb to .gdb
Update any SQL syntax Use load only mode to maximize data transfer performance
Dim pFeatureClassLoad As IFeatureClassLoad Set pFeatureClassLoad = pFeatureClass pFeatureClassLoad.LoadOnlyMode = True
No other differences in ArcObjects
File geodatabase at 9.2
Introduction
Comparisons and capabilities
Storage
File system operations, data types, storage limits and
requirements, configuration keywords
Compression Performance vs. other formats
Display, query, load, calculate Performance tips
Migrating to the file geodatabase Additional Information
For more information
More useful on-line help topics Use the Search tab to search for the topics
Try these:
Types of geodatabases Migrating to the file geodatabase How raster data is stored in a geodatabase Configuration keywords for file geodatabases Setting spatial indexes About compressing file geodatabase data Compacting file and personal geodatabases
