Importing Data

Introduction

Velocity provides powerful data import capabilities through CSV files, allowing you to efficiently load data into your database tables. The import functionality is flexible, supporting various data types including geospatial data, handling null values, custom date formats, and configurable column mappings.

Basic Import Operation

At its simplest, importing data involves three components:

A Table object defining the target schema
A CSV file containing the data to import
An ImportConfig object specifying how to interpret the CSV

Simple Import Example

using YndigoBlue.Velocity.Engine;
using YndigoBlue.Velocity.Model;
using YndigoBlue.Velocity.Data.PostgreSql;

var connection = new PostgreSqlDatasourceConnection
{
    Hostname = "localhost",
    Port = 5432,
    Username = "postgres",
    Password = "password",
    Database = "myapp"
};

using (var manager = new Manager(connection))
{
    // Load schema and get the table
    Schema schema = manager.LoadSchema("public");
    Table usersTable = schema["users"];

    // Import data with auto-generated configuration
    manager.ImportData(usersTable, "users.csv");
}

The ImportConfig Class

The ImportConfig class controls all aspects of how CSV data is interpreted and imported.

Properties

Property	Type	Description	Default (ISO)	Default (Culture)
`DateFormat`	`string`	Format string for Date columns	`"yyyy-MM-dd"`	Culture-specific
`DateTimeFormat`	`string`	Format string for DateTime columns	`"yyyy-MM-dd HH:mm:ss"`	Culture-specific
`Encoding`	`Encoding`	Encoding the file is read as. Unset reads it as UTF-8	unset	unset
`EscapeCharacter`	`char?`	Character escaping a quote inside a quoted field. Unset reads a doubled quote as one	unset	unset
`FieldDelimiter`	`char`	Character separating fields	`','`	Culture list separator
`IntervalFormat`	`string`	Format string for Interval columns	`"c"`	`"c"`
`KeepIdentity`	`bool`	Import values for autogenerate columns	`false`	`false`
`LineTerminator`	`string`	Sequence ending each row. Unset ends a row at either line ending	unset	unset
`Mappings`	`ImportConfigMappings`	Column mapping configuration	Empty	Empty
`QuoteCharacter`	`char`	Character for quoting fields	`'"'`	`'"'`
`TimeFormat`	`string`	Format string for Time columns	`"HH:mm:ss"`	Culture-specific
`TimestampFormat`	`string`	Format string for Timestamp columns	`"yyyy-MM-dd HH:mm:ss zzz"`	Culture-specific
`UseColumnHeaders`	`bool`	Whether CSV has header row	`true`	`true`

Constructor Options

You can create an ImportConfig using DateFormatType or auto-generate from a Table.

Default Constructor:

// Creates config with ISO date formats
var config = new ImportConfig();

With Date Format Type:

// ISO format (recommended for consistency)
var config = new ImportConfig(DateFormatType.Iso);

// Culture-specific format (uses current culture settings)
var config = new ImportConfig(DateFormatType.Culture);

Auto-Generate from Table:

// Automatically creates mappings for all non-autogenerated columns
var config = new ImportConfig(table);

// With date format type
var config = new ImportConfig(table, DateFormatType.Iso);

// With KeepIdentity flag
var config = new ImportConfig(table, keepIdentity: true);

// All options
var config = new ImportConfig(table, DateFormatType.Iso, keepIdentity: true);

Date and Time Formats

Velocity supports five temporal DataType values, each with its own configurable format string on ImportConfig:

Property	Applies to	ISO Default	Culture Default
`DateFormat`	`DataType.Date`	`"yyyy-MM-dd"`	`ShortDatePattern`
`DateTimeFormat`	`DataType.DateTime`	`"yyyy-MM-dd HH:mm:ss"`	`FullDateTimePattern`
`IntervalFormat`	`DataType.Interval`	`"c"`	`"c"`
`TimeFormat`	`DataType.Time`	`"HH:mm:ss"`	`ShortTimePattern`
`TimestampFormat`	`DataType.Timestamp`	`"yyyy-MM-dd HH:mm:ss zzz"`	`RFC1123Pattern`

ISO Format (Recommended)

var config = new ImportConfig(DateFormatType.Iso);
// DateFormat:      "yyyy-MM-dd"
// DateTimeFormat:  "yyyy-MM-dd HH:mm:ss"
// IntervalFormat:  "c"
// TimeFormat:      "HH:mm:ss"
// TimestampFormat: "yyyy-MM-dd HH:mm:ss zzz"

Date

Controls parsing of DataType.Date columns. ISO default is "yyyy-MM-dd".

CSV Example:

user_id,username,birthdate
1,john_doe,1990-05-15
2,jane_smith,1985-11-23

Custom Formats:

config.DateFormat = "MM/dd/yyyy";     // US format:   05/15/1990
config.DateFormat = "dd-MMM-yyyy";    // Abbreviated: 15-Jan-1990
config.DateFormat = "yyyyMMdd";       // Compact:     19900515

DateTime

Controls parsing of DataType.DateTime columns (date and time without a timezone offset). ISO default is "yyyy-MM-dd HH:mm:ss". Culture mode uses the system's FullDateTimePattern.

CSV Example:

event_id,event_name,event_time
1,Meeting,2024-01-15 14:30:00
2,Conference,2024-02-20 09:00:00

Custom Formats:

config.DateTimeFormat = "yyyy-MM-dd hh:mm:ss tt";    // 12-hour with AM/PM
config.DateTimeFormat = "dd/MM/yyyy HH:mm";           // European short format

Interval

Controls parsing of DataType.Interval columns, which represent a duration rather than a point in time. The default is "c" — the .NET constant (invariant) TimeSpan format — and is the same for both ISO and Culture modes.

The "c" format expects values in the form [-][d.]hh:mm:ss[.fffffff]:

Value	Meaning
`00:30:00`	30 minutes
`01:15:00`	1 hour 15 minutes
`1.02:03:04`	1 day, 2 hours, 3 minutes, 4 seconds
`-00:15:00`	Negative 15 minutes

CSV Example:

task_id,task_name,estimated_duration
1,Quick review,00:30:00
2,Full audit,4.08:00:00
3,Brief call,00:15:30

IntervalFormat rarely needs to be changed as "c" is universally portable across all supported databases.

Time

Controls parsing of DataType.Time columns. ISO default is "HH:mm:ss" (24-hour). Culture mode uses the system's ShortTimePattern.

CSV Example:

shift_id,label,start_time
1,Morning,07:30:00
2,Evening,18:00:00

Custom Formats:

config.TimeFormat = "hh:mm:ss tt";   // 12-hour with AM/PM: 07:30:00 AM
config.TimeFormat = "HH:mm";         // Hours and minutes only: 07:30

Timestamp

Controls parsing of DataType.Timestamp columns, which include a timezone offset. ISO default is "yyyy-MM-dd HH:mm:ss zzz". Culture mode uses the system's RFC1123Pattern.

The zzz specifier parses the timezone offset as +HH:mm or -HH:mm.

CSV Example:

recorded_at
2007-01-01 00:00:00 +01:00
2010-05-15 08:00:00 -02:00
2015-11-16 14:52:18 +05:00

Custom Formats:

config.TimestampFormat = "yyyy-MM-dd HH:mm:ss K";      // Timezone designator (Z or ±HH:mm)
config.TimestampFormat = "yyyy-MM-dd'T'HH:mm:ss'Z'";   // ISO 8601 UTC literal

Culture-Specific Formatting

When using DateFormatType.Culture, each format property is derived from the current thread's CultureInfo rather than a fixed ISO string. This is useful when importing files generated on the same system or in a known locale, but reduces portability across systems with different regional settings.

var config = new ImportConfig(DateFormatType.Culture);

The culture source for each property is:

Property	Culture Source	en-US Example	en-GB Example
`DateFormat`	`DateTimeFormat.ShortDatePattern`	`M/d/yyyy`	`dd/MM/yyyy`
`DateTimeFormat`	`DateTimeFormat.FullDateTimePattern`	`dddd, MMMM d, yyyy h:mm:ss tt`	`dd MMMM yyyy HH:mm:ss`
`IntervalFormat`	(invariant)	`c`	`c`
`TimeFormat`	`DateTimeFormat.ShortTimePattern`	`h:mm tt`	`HH:mm`
`TimestampFormat`	`DateTimeFormat.RFC1123Pattern`	`ddd, dd MMM yyyy HH':'mm':'ss 'GMT'`	`ddd, dd MMM yyyy HH':'mm':'ss 'GMT'`

Note

IntervalFormat is always "c" regardless of culture. TimestampFormat uses the RFC 1123 pattern which is the same across all cultures.

Use DateFormatType.Iso for files intended for cross-system use, automated processing, or round-tripping data exported by Velocity.

Column Mappings

Column mappings define the relationship between CSV headers and database columns. Velocity provides both automatic and manual mapping capabilities.

Automatic Mapping (No Configuration Required)

The simplest approach is to let Velocity automatically create the ImportConfig for you. When you don't provide an ImportConfig parameter, Velocity automatically generates one based on the table schema:

Table table = schema["users"];

// ImportConfig is automatically created with mappings for all non-autogenerated columns
manager.ImportData(table, "users.csv");

This is equivalent to:

Table table = schema["users"];
ImportConfig config = new ImportConfig(table);
manager.ImportData(table, "users.csv", config);

What gets mapped automatically:

All columns where AutoGenerate = false
Columns are mapped by name (CSV header matches column name exactly)
Data types are inferred from the table schema
ISO date format is used by default (DateFormatType.Iso)

Use automatic creation when:

Your CSV headers exactly match your database column names
You're using standard ISO date/time formats
You don't need to preserve identity values (KeepIdentity = false)
You're using default delimiters (comma) and quote characters

When KeepIdentity = true:

var config = new ImportConfig(table, keepIdentity: true);

// Now autogenerated columns are ALSO included in the mappings
// This allows importing specific identity values
manager.ImportData(table, "data_with_ids.csv", config);

When to Provide an Explicit ImportConfig

You need to explicitly create and pass an ImportConfig when you need to customize any import behavior:

// Need to preserve identity values
var config = new ImportConfig(table, keepIdentity: true);
manager.ImportData(table, "users.csv", config);

// Need custom date formats
var config = new ImportConfig(table, DateFormatType.Culture);
config.DateFormat = "MM/dd/yyyy";
manager.ImportData(table, "users.csv", config);

// Need different delimiter
var config = new ImportConfig(table);
config.FieldDelimiter = ';';
manager.ImportData(table, "european_data.csv", config);

// CSV has no headers
var config = new ImportConfig(table);
config.UseColumnHeaders = false;
manager.ImportData(table, "data_no_headers.csv", config);

// CSV headers don't match column names (see Manual Mapping below)

Manual Mapping

When CSV headers don't match column names, or you need selective import, use manual mappings:

var config = new ImportConfig(DateFormatType.Iso);

// Add individual mappings: (csvHeader, columnName, dataType)
config.Mappings.AddMapping("User ID", "user_id", DataType.Long);
config.Mappings.AddMapping("User Name", "username", DataType.VarChar);
config.Mappings.AddMapping("Email Address", "email", DataType.VarChar);
config.Mappings.AddMapping("Join Date", "created_at", DataType.DateTime);

CSV with different headers:

User ID,User Name,Email Address,Join Date
1,john_doe,john@example.com,2024-01-15 10:00:00
2,jane_smith,jane@example.com,2024-01-16 11:30:00

Programmatic Mapping

Build mappings programmatically for complex scenarios:

var config = new ImportConfig();
config.DateFormat = "yyyy-MM-dd";
config.FieldDelimiter = ',';

foreach (Column column in table.Columns)
{
    // Skip autogenerated columns unless KeepIdentity is true
    if (!column.AutoGenerate || config.KeepIdentity)
    {
        config.Mappings.AddMapping(column.Name, column.Name, column.DataType);
    }
}

Mapping Operations

The ImportConfigMappings class provides methods like AddMapping (both the header/call-order and the zero-based positional overloads), AddColumn, HeaderExists, ColumnExists, GetColumn, GetDataType, GetOrdinal, RemoveMapping, and Clear:

// Add a column directly from table definition
config.Mappings.AddColumn(column);

// Add a positional mapping for a headerless import (bind by zero-based source field)
config.Mappings.AddMapping(0, "user_id", DataType.Long);

// Check if header exists
if (config.Mappings.HeaderExists("username"))
{
    // ...
}

// Check if column is mapped
if (config.Mappings.ColumnExists("user_id"))
{
    // ...
}

// Get column name for header
string columnName = config.Mappings.GetColumn("User Name");

// Get data type for header
DataType dataType = config.Mappings.GetDataType("User Name");

// Get the zero-based source field a mapping reads from (headerless imports)
int ordinal = config.Mappings.GetOrdinal("User Name");

// Remove a mapping
config.Mappings.RemoveMapping("obsolete_header");

// Clear all mappings
config.Mappings.Clear();

Importing Geospatial Data

Velocity supports importing geospatial data using Well-Known Text (WKT) format in CSV files.

Supported Geometry Types

Both DataType.Geography and DataType.Geometry columns accept WKT format:

POINT - Single point location
LINESTRING - Connected line segments
POLYGON - Closed area
MULTIPOINT - Collection of points
MULTILINESTRING - Collection of lines
MULTIPOLYGON - Collection of polygons

Point Data Example

Table Definition:

Table locationsTable = schema.CreateTable("locations");
locationsTable.AddColumn("location_id", DataType.Long, autoGenerate: true, notNull: true);
locationsTable.AddColumn("name", DataType.VarChar, size: 100);
locationsTable.AddColumn("coordinates", DataType.Geography, notNull: true);
locationsTable.CreatePrimaryKey("pk_locations", "location_id");

// Create spatial index for performance
locationsTable.CreateSpatialIndex("spi_coordinates", "coordinates");

CSV File (locations.csv):

name,coordinates
SingleHop LLC,POINT(-91 38)
ZOHO,POINT(-94 38)
Google LLC,POINT(-100 35)
Amsterdam,POINT(-98 34)

Import Code:

var config = new ImportConfig(locationsTable);

using (var manager = new Manager(connection))
{
    manager.ImportData(locationsTable, "locations.csv", config);
}

Complex Geometry Example

CSV with Polygons:

region_id,region_name,boundary
1,Downtown,POLYGON((-122.4 37.8, -122.4 37.7, -122.3 37.7, -122.3 37.8, -122.4 37.8))
2,Airport,POLYGON((-122.5 37.6, -122.5 37.5, -122.4 37.5, -122.4 37.6, -122.5 37.6))

Import with Identity Values:

// Keep the identity values from CSV instead of auto-generating
var config = new ImportConfig(regionsTable, keepIdentity: true);

using (var manager = new Manager(connection))
{
    manager.ImportData(regionsTable, "regions.csv", config);
}

Geospatial Data Notes

WKT coordinates are specified as (longitude latitude) for geographic data
Point format: POINT(longitude latitude)
Always enclose complex WKT in quotes if it contains commas
Use spatial indexes for optimal query performance
Geometry types support various coordinate reference systems

Importing Blob Data

Velocity supports two strategies for importing binary data (DataType.Blob) from a CSV file. Configure blob handling per column using SetBlobColumnBase64 or SetBlobColumnFile on ImportConfig.

Blob columns must be included in the column mappings just like any other column. If no blob configuration is registered for a column, any value in that CSV cell is discarded and the column is stored as NULL.

Base64 Encoding

When the CSV cell contains a Base64-encoded string, call SetBlobColumnBase64:

Table attachmentsTable = schema["attachments"];
var config = new ImportConfig(attachmentsTable);
config.SetBlobColumnBase64("Data");

using (var manager = new Manager(connection))
{
    manager.ImportData(attachmentsTable, "attachments.csv", config);
}

CSV format — the Data cell contains the raw Base64 string:

AttachmentID,Name,Data
1,logo.png,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==
2,readme.txt,SGVsbG8gV29ybGQ=

File-Based Blobs

When the blob data lives in a separate file on disk, call SetBlobColumnFile with a path pattern. The pattern may reference any other column in the CSV using {ColumnName} tokens. Paths are resolved relative to the directory containing the CSV file.

Table documentsTable = schema["documents"];
var config = new ImportConfig(documentsTable);
config.SetBlobColumnFile("Data", "files/{Category}/{Filename}");

using (var manager = new Manager(connection))
{
    manager.ImportData(documentsTable, "documents.csv", config);
}

CSV format — the Data cell is empty; the file is located via the pattern using Category and Filename values from the same row:

DocumentID,Category,Filename,Data
1,reports,q1_summary.pdf,
2,images,product_shot.jpg,
3,archives,backup.zip,

Given the CSV is at ./data/documents.csv, the files are expected at:

./data/files/reports/q1_summary.pdf
./data/files/images/product_shot.jpg
./data/files/archives/backup.zip

If a referenced file does not exist the column is stored as NULL for that row.

Pattern Token Reference

Any column present in the CSV can be used as a token in the pattern:

Token	Replaced with
`{ColumnName}`	The string value of that column in the current row

Tokens are replaced case-insensitively. Non-existent column tokens are left as-is.

Mixing Strategies

Different blob columns in the same table can use different strategies:

var config = new ImportConfig(mediaTable);
config.SetBlobColumnBase64("Thumbnail");       // small preview — inline Base64
config.SetBlobColumnFile("FullImage", "media/{Category}/{Filename}");  // large file on disk

manager.ImportData(mediaTable, "media.csv", config);

CSV format:

MediaID,Category,Filename,Thumbnail,FullImage
1,photos,sunset.jpg,/9j/4AAQSkZJRgAB...,

Thumbnail is decoded from the Base64 cell; FullImage is read from ./media/photos/sunset.jpg.

Teradata Bulk Import: Staging Tables

Teradata uses a multi-phase staging strategy for bulk imports to work around driver limitations with large Clob and geospatial columns. Understanding this mechanism is important when tuning import performance or troubleshooting Teradata-specific issues.

How It Works

Phase 1 – Parallel staging inserts. The source data is split into N batches (controlled by NumberOfStagingTables, default 4). Each batch is inserted concurrently into its own temporary staging table ({table}_{N}_IMP). By default, Clob columns are retyped to VARCHAR(VarCharCastSize) and geospatial columns are retyped to VARCHAR(VarCharCastSize) holding WKT text. Either or both casts can be disabled — see Controlling Clob and Geospatial Casting below.
Phase 2 – Staging-to-target merge. A single INSERT INTO target SELECT ... FROM staging statement is executed for each staging table. If geospatial columns were cast to VARCHAR in Phase 1, the merge wraps them in SYSSPATIAL.ST_GeomFromText() to convert back to the native spatial type. This phase runs within an optional transaction controlled by ImportTransaction.
Phase 3 – Cleanup. All staging tables are dropped regardless of success or failure.

Controlling Clob and Geospatial Casting

Three properties on TeradataDatasourceConnection govern how Clob and geospatial columns are handled during import:

Property	Default	Effect
`ImportCastClob`	`true`	Clob columns in staging tables are retyped as `VARCHAR(VarCharCastSize)`
`ImportCastGeo`	`true`	Geometry/Geography columns in staging tables are retyped as `VARCHAR(VarCharCastSize)` (WKT text). The Phase 2 merge wraps these columns in `ST_GeomFromText()` to convert back to the native spatial type.
`VarCharCastSize`	`32000`	The VARCHAR length applied to every column that is cast — both Clob and geospatial

The two flags are independent and each only affects its own column type. The following matrix shows the resulting staging-table type for each combination:

`ImportCastGeo`	`ImportCastClob`	Geo/Geography staging type	Clob staging type	`VarCharCastSize` limit applies to
`true`	`true`	`VARCHAR(VarCharCastSize)`	`VARCHAR(VarCharCastSize)`	Both Geo and Clob columns
`true`	`false`	`VARCHAR(VarCharCastSize)`	Clob (native, no size limit)	Geo columns only
`false`	`true`	Native spatial type (no size limit)	`VARCHAR(VarCharCastSize)`	Clob columns only
`false`	`false`	Native spatial type (no size limit)	Clob (native, no size limit)	Neither — no VARCHAR casts applied

Note

When ImportCastGeo is false the geospatial columns remain as their native spatial type in the staging table and the Phase 2 merge passes them through without a ST_GeomFromText() call. This removes the VarCharCastSize constraint for geospatial data entirely but requires the Teradata driver to handle the native spatial type during the bulk load directly.

Choosing the right strategy for large data

WKT fits within VarCharCastSize — leave both flags at their defaults (true). The VARCHAR cast is the fastest bulk-load path for the Teradata driver.
WKT exceeds VarCharCastSize but you want to keep the cast — increase VarCharCastSize to a value larger than your longest WKT string:

var connection = new TeradataDatasourceConnection
{
    Hostname = "teradata.example.com",
    Port = 1025,
    Username = "dbc",
    Password = "password",
    Database = "myapp_db"
};

connection.VarCharCastSize = 65000;   // Increase for large polygons
connection.NumberOfStagingTables = 4;

using (var manager = new Manager(connection))
{
    Schema schema = manager.LoadSchema("myschema");
    Table regionsTable = schema["regions"];

    manager.ImportData(regionsTable, "large_polygons.csv");
}

WKT exceeds the maximum VARCHAR size the database supports — set ImportCastGeo = false to keep geospatial columns in their native spatial type and bypass the VARCHAR limit entirely:

connection.ImportCastGeo = false;     // Geo columns stay as native spatial — no VARCHAR size limit
connection.ImportCastClob = true;     // Clob columns can still be cast independently

Clob data exceeds VarCharCastSize — set ImportCastClob = false. Clob columns remain as native Clob (unbounded) in the staging table:

connection.ImportCastClob = false;    // Clob columns stay as native Clob — no VARCHAR size limit
connection.ImportCastGeo = true;      // Geo columns can still be cast independently

Transaction Control

By default (ImportTransaction = true) the staging-to-target merge in Phase 2 runs inside a single transaction. If the merge fails, all staging inserts for that batch are rolled back. Set ImportTransaction = false to skip the wrapping transaction when importing into already-empty tables or when external transaction management is preferred.

Note

ImportTransaction is a universal setting available on all database connection types, but its effect is most significant on Teradata because the merge phase operates across multiple staging tables in one pass.

Streaming Large Files

By default ImportData parses the whole CSV into memory before sending any of it to the database, so peak memory use scales with the size of the file. A 100 MB file costs several hundred megabytes of heap in the process doing the import.

Passing streamImport: true reads the file in chunks and sends each one as it is read, so only one chunk is ever held and peak memory follows the chunk rather than the file.

using (var manager = new Manager(connection))
{
    Schema schema = manager.LoadSchema("public");
    Table eventsTable = schema["events"];

    // Reads and sends the file in chunks instead of loading it in full
    manager.ImportData(eventsTable, "events_10m_rows.csv", config, streamImport: true);
}

The chunk size is the existing ImportBatchSize connection setting, so it is tuned the same way as any other import batching:

connection.ImportBatchSize = 50000;   // Larger chunks: fewer round trips, more memory per chunk

The setting counts rows, not bytes, so what a chunk costs depends on how wide the rows are. The default of 10,000 suits ordinary rows. A file whose rows carry a WKT geometry or a large text column can run to tens of kilobytes each, and wants a few hundred rows to a chunk instead:

connection.ImportBatchSize = 500;     // Rows carrying large geometries or text

What Streaming Does Not Change

The rows that end up in the table. A streamed import writes exactly what a whole-file import writes, in the same order, with the same mappings and type conversions.
The import config. Every ImportConfig option — mappings, delimiters, date formats, blob handling, KeepIdentity — behaves identically.
Transaction behaviour. Streaming gives up nothing a whole-file import guaranteed; ImportTransaction means the same thing either way. The one cost is that an import holds its transaction, and whatever locks that implies, for as long as the file takes to read.

When to Use It

Use streamImport: true for files large enough that holding them in memory matters — as a rule of thumb, anything above a few tens of megabytes, or any file whose size you do not control. For small files the default is marginally faster because it avoids the per-chunk overhead, and the memory saving is not worth measuring.

Importing GeoJSON

Warning

GeoJSON import is not available in the Community Edition of Velocity. It requires the Full Edition.

ImportGeoJSON loads a GeoJSON file into a single table. Features are appended, exactly as CSV rows are — the table is not emptied first.

A Feature has three parts, and each is mapped separately with GeoJSONImportConfig:

Part of the Feature	Mapped with	Notes
`geometry`	`MapGeometry`	Defaults to `Geography`, since GeoJSON coordinates are WGS 84 longitude/latitude
`id`	`MapId`	Has its own mapping because it is a member of the Feature, not one of its properties
`properties`	`MapProperty`	Addressed by name, including nested names

using (var manager = new Manager(connection))
{
    Schema schema = manager.LoadSchema("public");
    Table sitesTable = schema["sites"];

    var config = new GeoJSONImportConfig();

    config.MapGeometry("location", DataType.Geography);
    config.MapId("site_id", DataType.Long);
    config.MapProperty("location_name", "name", DataType.VarChar);
    config.MapProperty("capacity", "capacity", DataType.Integer);

    manager.ImportGeoJSON(sitesTable, "sites.geojson", config);
}

Nested Properties

A property of a nested object is addressed with a dotted path. Given this feature:

{
  "type": "Feature",
  "geometry": { "type": "Point", "coordinates": [-122.4194, 37.7749] },
  "properties": {
    "location_name": "San Francisco Hub",
    "contact_info": {
      "manager": "Jane Doe",
      "phone": "555-0199"
    },
    "operating_hours": {
      "weekdays": "08:00-20:00",
      "weekends": "10:00-16:00"
    }
  }
}

each nested value is mapped to its own column:

config.MapProperty("contact_info.manager", "manager", DataType.VarChar);
config.MapProperty("contact_info.phone", "phone", DataType.VarChar);
config.MapProperty("operating_hours.weekdays", "hours_weekday", DataType.VarChar);

Paths may be nested to any depth — contact_info.escalation.name is as valid as contact_info.manager.

A path that does not resolve imports as null rather than failing. That covers a missing property, a missing object part-way along the path, and a property that is present but JSON null, so features that omit optional details import alongside those that carry them without special handling.

Unmapped Columns

Columns the table has but the configuration does not map are left to the database — they take their default, or null — exactly as in a CSV import. Properties in the file that nothing maps are ignored.

Streaming

ImportGeoJSON takes the same streamImport flag as ImportData, with the same meaning: the file is read in chunks of ImportBatchSize features rather than parsed into memory whole. See Streaming Large Files.

manager.ImportGeoJSON(sitesTable, "sites_1m_features.geojson", config, streamImport: true);

Supported Documents

Every object GeoJSON allows at the top level is accepted:

Document	Rows imported
`FeatureCollection`	One per feature
`Feature`	One
`Point`, `MultiPoint`, `LineString`, `MultiLineString`, `Polygon`, `MultiPolygon`, `GeometryCollection`	One

A bare geometry has neither an id nor properties, so only a geometry mapping draws anything from it and every other mapped column imports as null. A Feature with "geometry": null imports its properties with a null geometry.

A document is treated as a collection only when the root object itself carries a features array, so a property that merely happens to share the name — one inside a Feature's properties, say — is never mistaken for it. Members may appear in any order, and an empty features array imports nothing without error.

A top-level JSON array, or an object whose type is not one of the above, is rejected with an InvalidDataException.

Note

DB2 cannot store a GeometryCollection, so a document of that type cannot be imported there. This is a property of DB2 rather than of GeoJSON, and applies equally to a CSV import of the same shape.

Data Types

Every data type a CSV import can target is also carried from GeoJSON. JSON has only one number type and no date type, so numbers arrive as decimals and dates as strings; both are converted to the column's type on the way in.

Velocity type	Written in GeoJSON as
`Boolean`	`true` / `false`, or a string containing one
`Byte`, `Short`, `Integer`, `Long`	A number, or a string containing one
`Float`, `Double`, `Decimal`	A number, or a string containing one
`Char`, `VarChar`, `Clob`	A string
`Blob`	A Base64 string
`Date`	A string matching `DateFormat`
`Time`	A string matching `TimeFormat`
`DateTime`	A string matching `DateTimeFormat`
`Timestamp`	A string matching `TimestampFormat`
`Interval`	A string matching `IntervalFormat`
`Geography`, `Geometry`	The Feature's `geometry`, mapped with `MapGeometry`

The date and number settings are the ones ImportConfig already exposes, and GeoJSONImportConfig inherits them — so the constructor's DateFormatType choice, the individual format strings, and NumberSeparator all behave exactly as they do for a CSV import:

// ISO formats by default; pass DateFormatType.Culture for the current culture's patterns
var config = new GeoJSONImportConfig(DateFormatType.Iso);

config.DateFormat = "dd/MM/yyyy";     // Override any individual format
config.NumberSeparator = ",";         // Strip thousands separators before parsing

A value that cannot be converted to its column's type raises an InvalidImportMappingsException naming the column, rather than being silently dropped.

String, Query, Expression and Guid cannot be imported. These are not storage types — they are used elsewhere in the framework for query construction — and ImportConfig rejects them for CSV imports on the same grounds.

Limitations

Arrays are not supported. A property holding an array, or a path that runs through one, imports as null. Only objects can be traversed.

Handling Autogenerated Columns (Identity)

By default, Velocity skips autogenerated columns during import, allowing the database to generate values automatically. The ImportData method respects the KeepIdentity setting in ImportConfig.

Default Behavior (KeepIdentity = false)

Table table = schema.CreateTable("products");
table.AddColumn("product_id", DataType.Long, autoGenerate: true, notNull: true);
table.AddColumn("product_name", DataType.VarChar, size: 100);
table.AddColumn("price", DataType.Decimal, precision: 10, scale: 2);

var config = new ImportConfig(table);  // KeepIdentity defaults to false

// CSV should NOT include product_id - it will be auto-generated

CSV File (products.csv):

product_name,price
Widget,19.99
Gadget,29.99

Importing Identity Values (KeepIdentity = true)

When you need to preserve specific identity values (e.g., migrating data, maintaining foreign key relationships):

var config = new ImportConfig(table, keepIdentity: true);

using (var manager = new Manager(connection))
{
    manager.ImportData(table, "products_with_ids.csv", config);

    // After import, new records will continue from the highest imported ID
    var record = new Record(table);
    record.SetFieldVarChar("product_name", "New Product");
    record.SetFieldDecimal("price", 39.99M);

    long newId = manager.AddRecord(record);
    // newId will be next available ID after imported data
}

CSV File (products_with_ids.csv):

product_id,product_name,price
100,Widget,19.99
101,Gadget,29.99
102,Doohickey,39.99

Files With and Without Headers

With Headers (Default)

var config = new ImportConfig(table);
config.UseColumnHeaders = true;  // Default value

manager.ImportData(table, "data_with_headers.csv", config);

CSV File:

user_id,username,email,created_at
1,john_doe,john@example.com,2024-01-15
2,jane_smith,jane@example.com,2024-01-16

Without Headers

When your CSV file has no header row, columns are bound to database columns by position rather than by name. There are two ways to declare a positional mapping.

Positional by call order (string overload)

Each AddMapping(header, columnName, dataType) call binds the next source field, in the order the calls are made — the first call takes field 0, the second field 1, and so on. In this mode the first argument is only a placeholder: its value is never matched against the file (there is no header row), but it must be unique across mappings. Add the mappings in the exact order the fields appear in the file:

var config = new ImportConfig();
config.UseColumnHeaders = false;
config.FieldDelimiter  = ',';

// Bound by call order: field 0 → FacilityID, field 1 → Name, ...
config.Mappings.AddMapping("FacilityID", "FacilityID", DataType.Integer);
config.Mappings.AddMapping("Name", "Name", DataType.VarChar);
config.Mappings.AddMapping("Address", "Address", DataType.VarChar);
config.Mappings.AddMapping("City", "City", DataType.VarChar);
config.Mappings.AddMapping("Latitude", "Latitude", DataType.Float);
config.Mappings.AddMapping("Longitude", "Longitude", DataType.Float);

manager.ImportData(table, "data_no_headers.csv", config);

CSV File (no header row):

1001,Main Hospital,123 Main St,Springfield,42.123,-72.456
1002,City Clinic,456 Oak Ave,Riverside,41.234,-71.567
1003,County Medical,789 Elm Rd,Brookfield,40.345,-70.678

Positional by explicit index (integer overload)

AddMapping also has an overload that takes a zero-based source field index instead of a placeholder name:

void AddMapping(int columnNumber, string columnName, DataType columnType)

Because the source position is stated explicitly rather than inferred from call order, you can skip source fields or read them out of order — neither of which is possible with the call-order form. For example, given a tab-separated file whose fields are word_id, word, frequency, this imports only word and frequency, skipping the leading word_id field so the table's own key can auto-generate:

var config = new ImportConfig();
config.UseColumnHeaders = false;
config.FieldDelimiter  = '\t';

config.Mappings.AddMapping(1, "word", DataType.VarChar);       // source field 1
config.Mappings.AddMapping(2, "frequency", DataType.Integer);  // source field 2
// source field 0 (word_id) is never read

manager.ImportData(table, "word_frequency.txt", config);

The two forms are equivalent when fields are declared in order — a string AddMapping added k-th behaves exactly like AddMapping(k, ...).

Partial-column imports

The mapped columns do not have to cover every column in the table. Any table column that is not mapped is imported as NULL (or its column default). This is the natural counterpart to skipping source fields: the file may carry fewer columns than the table, with the remaining columns populated by a later step. For example, importing (word_id, word, frequency) into a table that also defines computed initial_letter and length columns leaves those two NULL on insert.

Note

The integer columnNumber applies only to headerless imports. When UseColumnHeaders = true, columns are bound by matching the header row and the ordinal is ignored. As always, any column left unmapped must be nullable (or have a default); mapping no value to a notNull column without a default will fail on insert.

Handling Null Values

Velocity automatically handles null values in CSV imports. Empty fields or fields containing database-specific null representations are imported as NULL.

Null Value Examples

CSV with Nulls:

user_id,username,email,phone,last_login
1,john_doe,john@example.com,555-1234,2024-01-15 10:00:00
2,jane_smith,jane@example.com,,
3,bob_jones,,555-5678,2024-01-14 15:30:00

In this example:

Row 2: phone and last_login are NULL (empty fields)
Row 3: email is NULL (empty field)

Import Code for Nullable Columns

Table table = schema.CreateTable("users");
table.AddColumn("user_id", DataType.Long, autoGenerate: true, notNull: true);
table.AddColumn("username", DataType.VarChar, size: 50, notNull: true);
table.AddColumn("email", DataType.VarChar, size: 100);      // Nullable
table.AddColumn("phone", DataType.VarChar, size: 20);       // Nullable
table.AddColumn("last_login", DataType.DateTime);           // Nullable

var config = new ImportConfig(table);

using (var manager = new Manager(connection))
{
    manager.ImportData(table, "users_with_nulls.csv", config);
}

All Data Types with Nulls

Velocity supports null values for all nullable column types:

CSV Example:

BooleanColumn,ByteColumn,IntegerColumn,LongColumn,FloatColumn,DoubleColumn,DecimalColumn,DateColumn,DateTimeColumn,TimestampColumn,VarCharColumn,GeometryColumn
TRUE,1,100,1000,10.5,100.5,1000.50,2024-01-01,2024-01-01 10:00:00,2024-01-01 10:00:00 +00:00,Sample,POINT(1 1)
FALSE,,,,,,,,,,No nulls here,POINT(2 2)
,2,200,2000,20.5,200.5,2000.50,,,,Has some nulls,

Important:

Columns defined with notNull: true cannot accept null values during import
Empty strings in VARCHAR/CHAR columns are treated as empty strings, not NULL
For explicit NULL in string columns, the field must be completely empty

Field Delimiters and Special Characters

Custom Delimiters

var config = new ImportConfig();

// Tab-delimited
config.FieldDelimiter = '\t';

// Pipe-delimited
config.FieldDelimiter = '|';

// Semicolon-delimited (common in European locales)
config.FieldDelimiter = ';';

Quoted Fields

Fields containing delimiters, quotes, or newlines should be quoted:

config.QuoteCharacter = '"';

Set EscapeCharacter only for files in which a quote inside a value is preceded by a character rather than doubled:

// Only for a file that writes \" rather than ""
config.EscapeCharacter = '\\';

CSV with Quoted Fields:

id,name,description
1,"Smith, John","This product includes a feature called ""Advanced Mode"""
2,Basic Widget,"Simple
Multi-line
Description"

A quote inside a quoted field is written twice, and a value may run across lines for as long as it stays quoted. A file whose quotes do not balance is rejected with an InvalidDataException rather than being guessed at.

Line Terminators

A row ends at either a line feed or a carriage return and line feed unless LineTerminator is set, so files written either way are read without configuration.

Set LineTerminator only when rows are separated by a sequence that is not a line break. A configured value is matched exactly, so a file written with a different terminator is then read as a single row.

// Rows separated by something that is not a line break at all
config.LineTerminator = "|END|";

Encoding

The file is read as UTF-8 unless Encoding is set.

A byte order mark takes precedence over Encoding. A file beginning with a UTF-8, UTF-16 or UTF-32 mark is read as that encoding regardless of the configured value.

Encoding is required for files that are not UTF-8 and contain no byte order mark, typically legacy single-byte code pages. Such files carry no indication of their encoding. Read as UTF-8, every byte above 127 is decoded as U+FFFD and the import completes without error.

// A file written in Latin-1 rather than UTF-8
config.Encoding = Encoding.Latin1;

Code pages beyond the ones built in have to be registered before Encoding.GetEncoding will return them:

Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
config.Encoding = Encoding.GetEncoding(1252);

Complete Examples

Example 1: Basic Import with Automatic Configuration

This example demonstrates using Manager with SqlServerDatasourceConnection to load a Schema and import data.

using YndigoBlue.Velocity.Engine;
using YndigoBlue.Velocity.Model;
using YndigoBlue.Velocity.Data.SqlServer;
using YndigoBlue.Velocity.Enums;

var connection = new SqlServerDatasourceConnection
{
    Hostname = "localhost",
    Port = 1433,
    Username = "sa",
    Password = "password",
    Database = "products_db"
};

using (var manager = new Manager(connection))
{
    // Load the schema
    Schema schema = manager.LoadSchema("dbo");
    Table productsTable = schema["products"];

    // Import with automatic configuration - ImportConfig is created automatically
    manager.ImportData(productsTable, "products.csv");

    Console.WriteLine("Import completed successfully");
}

Example 2: Import with Custom Date Formats

This example shows custom date formats with DateFormatType.

var connection = new MySqlDatasourceConnection
{
    Hostname = "localhost",
    Port = 3306,
    Username = "root",
    Password = "password",
    Database = "events_db"
};

using (var manager = new Manager(connection))
{
    Schema schema = manager.LoadSchema("events_db");
    Table eventsTable = schema["events"];

    // Create config with custom date formats
    var config = new ImportConfig(eventsTable, DateFormatType.Iso);
    config.DateFormat      = "MM/dd/yyyy";
    config.DateTimeFormat  = "MM/dd/yyyy hh:mm tt";  // US format with AM/PM
    config.IntervalFormat  = "c";
    config.TimeFormat      = "hh:mm tt";
    config.TimestampFormat = "MM/dd/yyyy hh:mm tt zzz";

    manager.ImportData(eventsTable, "events_us_format.csv", config);
}

Example 3: Import with Manual Column Mapping

This example uses manual mappings with AddMapping.

using (var manager = new Manager(connection))
{
    Schema schema = manager.LoadSchema("public");
    Table customersTable = schema["customers"];

    // Create config with manual mappings
    var config = new ImportConfig(DateFormatType.Iso);

    // CSV headers don't match database column names
    config.Mappings.AddMapping("Customer ID", "customer_id", DataType.Long);
    config.Mappings.AddMapping("Full Name", "customer_name", DataType.VarChar);
    config.Mappings.AddMapping("E-mail", "email", DataType.VarChar);
    config.Mappings.AddMapping("Registration", "registered_at", DataType.DateTime);

    manager.ImportData(customersTable, "customers_export.csv", config);
}

Example 4: Import Geospatial Data with Identity Values

This example demonstrates PostgreSqlDatasourceConnection with geospatial data using Schema and Table.

using YndigoBlue.Velocity.Model;
using YndigoBlue.Velocity.Enums;

var connection = new PostgreSqlDatasourceConnection
{
    Hostname = "localhost",
    Port = 5432,
    Username = "postgres",
    Password = "password",
    Database = "gis_db"
};

using (var manager = new Manager(connection))
{
    // Create schema with geospatial table
    Schema schema = Schema.NewSchema("locations");
    Table locationsTable = schema.CreateTable("points_of_interest");

    locationsTable.AddColumn("poi_id", DataType.Long, autoGenerate: true, notNull: true);
    locationsTable.AddColumn("name", DataType.VarChar, size: 200, notNull: true);
    locationsTable.AddColumn("location", DataType.Geography, notNull: true);
    locationsTable.CreatePrimaryKey("pk_poi", "poi_id");
    locationsTable.CreateSpatialIndex("spi_poi_location", "location");

    // Deploy schema
    manager.BuildSchema(schema, overwrite: true);

    // Import with identity preservation
    var config = new ImportConfig(locationsTable, keepIdentity: true);

    manager.ImportData(locationsTable, "poi_data.csv", config);

    Console.WriteLine("Geospatial data imported with preserved IDs");
}

CSV File (poi_data.csv):

poi_id,name,location
1,Golden Gate Bridge,POINT(-122.4783 37.8199)
2,Statue of Liberty,POINT(-74.0445 40.6892)
3,Space Needle,POINT(-122.3493 47.6205)
4,Willis Tower,POINT(-87.6359 41.8789)

Example 5: Import File Without Headers

This example shows importing CSV files without headers using LoadSchema.

using (var manager = new Manager(connection))
{
    Schema schema = manager.LoadSchema("analytics");
    Table metricsTable = schema["daily_metrics"];

    var config = new ImportConfig();
    config.UseColumnHeaders = false;
    config.DateFormat      = "yyyy-MM-dd";
    config.DateTimeFormat  = "yyyy-MM-dd HH:mm:ss";
    config.FieldDelimiter  = ',';
    config.IntervalFormat  = "c";
    config.TimeFormat      = "HH:mm:ss";
    config.TimestampFormat = "yyyy-MM-dd HH:mm:ss zzz";

    // Define mappings in column order as they appear in CSV
    for (int i = 0; i < metricsTable.Columns.Count; i++)
    {
        Column column = metricsTable[i];
        if (!column.AutoGenerate)
        {
            config.Mappings.AddMapping(column.Name, column.Name, column.DataType);
        }
    }

    manager.ImportData(metricsTable, "metrics_no_headers.csv", config);
}

Example 6: Large File Import with Mixed Data Types

This example uses ImportConfig with timestamp formats.

using (var manager = new Manager(connection))
{
    Schema schema = manager.LoadSchema("analytics");
    Table sensorDataTable = schema["sensor_readings"];

    var config = new ImportConfig(sensorDataTable, DateFormatType.Iso);
    config.DateFormat      = "yyyy-MM-dd";
    config.DateTimeFormat  = "yyyy-MM-dd HH:mm:ss";
    config.IntervalFormat  = "c";
    config.TimeFormat      = "HH:mm:ss";
    config.TimestampFormat = "yyyy-MM-dd HH:mm:ss zzz";

    // Import large file
    manager.ImportData(sensorDataTable, "sensor_data_2024.csv", config);

    Console.WriteLine("Large dataset imported successfully");
}

CSV File (sensor_data_2024.csv):

sensor_id,reading_timestamp,temperature,humidity,location
S001,2024-01-01 00:00:00 +00:00,22.5,65.3,POINT(-122.4 37.8)
S001,2024-01-01 01:00:00 +00:00,22.3,66.1,POINT(-122.4 37.8)
S002,2024-01-01 00:00:00 +00:00,21.8,68.5,POINT(-118.2 34.1)
S002,2024-01-01 01:00:00 +00:00,21.9,67.9,POINT(-118.2 34.1)

Example 7: Import with Tab Delimiter and European Format

var connection = new SqliteDatasourceConnection
{
    FilePath = "./data/sales.db"
};

using (var manager = new Manager(connection))
{
    Schema schema = manager.LoadSchema("main");
    Table salesTable = schema["sales"];

    var config = new ImportConfig(salesTable, DateFormatType.Iso);
    config.DateFormat      = "dd.MM.yyyy";
    config.DateTimeFormat  = "dd.MM.yyyy HH:mm:ss";
    config.FieldDelimiter  = ';';              // Common in European CSV
    config.IntervalFormat  = "c";
    config.TimeFormat      = "HH:mm:ss";
    config.TimestampFormat = "dd.MM.yyyy HH:mm:ss zzz";

    manager.ImportData(salesTable, "european_sales.csv", config);
}

CSV File (european_sales.csv):

sale_id;customer_name;sale_date;amount
1;Müller GmbH;15.01.2024;1234,56
2;Société Générale;16.01.2024;2345,67

Best Practices

1. Use ISO Date Formats for Portability

// Recommended: ISO format works across all systems
var config = new ImportConfig(table, DateFormatType.Iso);

ISO formats (DateFormatType.Iso) are unambiguous and work consistently across different database systems and locales.

2. Validate Data Before Import

// Check if file exists
if (!File.Exists(csvFilePath))
{
    throw new FileNotFoundException($"CSV file not found: {csvFilePath}");
}

// Verify file is not empty
var fileInfo = new FileInfo(csvFilePath);
if (fileInfo.Length == 0)
{
    throw new InvalidOperationException("CSV file is empty");
}

3. Use Automatic Configuration When Possible

Let ImportData create the ImportConfig automatically when possible:

// Best: Let Velocity create the ImportConfig automatically
manager.ImportData(table, "data.csv");

// Good: Explicit config when you need it
var config = new ImportConfig(table);
manager.ImportData(table, "data.csv", config);

// Avoid: Manual mapping when not necessary
var config = new ImportConfig();
foreach (var column in table.Columns)
{
    config.Mappings.AddMapping(...);  // Unnecessary work
}

4. Handle Large Files Appropriately

For very large CSV files, consider:

Passing streamImport: true so the file is read in chunks instead of loaded whole — see Streaming Large Files
Importing during off-peak hours
Using database-specific bulk load features
Monitoring transaction log growth
Ensuring adequate disk space

manager.ImportData(table, "events_10m_rows.csv", config, streamImport: true);

5. Preserve Identity Values Only When Necessary

// Use KeepIdentity sparingly - only for migrations or specific requirements
var config = new ImportConfig(table, keepIdentity: true);

Most scenarios should allow the database to auto-generate identity values.

6. Test Imports on Sample Data First

// Test with a small subset first
using (var manager = new Manager(connection))
{
    // Import sample
    manager.ImportData(table, "sample_100_rows.csv", config);

    // Verify results
    var query = new Query().SelectAll(table);
    var results = manager.Retrieve(query);

    // If successful, import full dataset
    manager.ImportData(table, "full_dataset.csv", config);
}

7. Use Transactions for Data Integrity

The ImportData method wraps the import in a transaction by default (ImportTransaction = true), ensuring all-or-nothing behavior:

using (var manager = new Manager(connection))
{
    try
    {
        manager.ImportData(table, "data.csv", config);
        // All data imported successfully
    }
    catch (Exception ex)
    {
        // Transaction automatically rolled back
        Console.WriteLine($"Import failed: {ex.Message}");
    }
}

Set ImportTransaction = false on the connection when you are managing transactions externally or when the target table is guaranteed to be empty and a rollback is unnecessary. On Teradata this specifically controls whether the staging-to-target merge runs inside a transaction.

8. Document Your Import Configurations

Create reusable ImportConfig methods for consistency:

// Create reusable import configurations
public static class ImportConfigurations
{
    public static ImportConfig ForCustomerData(Table customersTable)
    {
        var config = new ImportConfig(customersTable, DateFormatType.Iso);
        config.DateFormat      = "yyyy-MM-dd";
        config.DateTimeFormat  = "yyyy-MM-dd HH:mm:ss";
        config.IntervalFormat  = "c";
        config.TimeFormat      = "HH:mm:ss";
        config.TimestampFormat = "yyyy-MM-dd HH:mm:ss zzz";
        return config;
    }

    public static ImportConfig ForEuropeanSales(Table salesTable)
    {
        var config = new ImportConfig(salesTable);
        config.DateFormat      = "dd.MM.yyyy";
        config.DateTimeFormat  = "dd.MM.yyyy HH:mm:ss";
        config.FieldDelimiter  = ';';
        config.IntervalFormat  = "c";
        config.TimeFormat      = "HH:mm:ss";
        config.TimestampFormat = "dd.MM.yyyy HH:mm:ss zzz";
        return config;
    }
}

Troubleshooting

Common Import Errors

Error: "Invalid Import Mappings"

Ensure all non-nullable columns are mapped
Verify column names match exactly (case-sensitive)
Check that data types are compatible

Error: "Value Set For Auto Generated Column"

Set KeepIdentity = true if you need to import identity values
Or remove the autogenerated column from your CSV

Error: "Date/Time Format Exception"

Verify DateFormat, TimeFormat, DateTimeFormat, IntervalFormat, or TimestampFormat matches your CSV data
Check for timezone information in timestamp columns

Error: "Geometry Parse Error"

Ensure WKT format is correct (e.g., POINT(lon lat))
Quote complex geometries that contain commas
Verify coordinate order (longitude, then latitude for geographic data)

Error: "Null Value in NOT NULL Column"

Check for empty fields in required columns
Verify CSV data completeness
Ensure notNull: true columns always have values

Summary

Velocity's import functionality provides:

Zero-Configuration Imports - Simply call ImportData for standard imports
Flexible Configuration - Control every aspect of CSV interpretation when needed
Automatic Mapping - ImportConfig created automatically from table schema
Type Safety - Proper handling of all DataType values including spatial data
Format Support - Customizable DateFormatType formats and delimiters
Identity Control - Choose whether to preserve or generate identity values
Null Handling - Automatic null value detection and import
Transaction Safety - All-or-nothing import with automatic rollback on error
Blob Support - Import binary data from inline Base64 strings or external files referenced by path patterns

By following these guidelines and examples, you can efficiently import data from CSV files into any supported database system with confidence and consistency.

Table of Contents

Importing Data

Introduction

Basic Import Operation

Simple Import Example

The ImportConfig Class

Properties

Constructor Options

Date and Time Formats

ISO Format (Recommended)

Date

DateTime

Interval

Time

Timestamp

Culture-Specific Formatting

Note

Column Mappings

Automatic Mapping (No Configuration Required)

When to Provide an Explicit ImportConfig

Manual Mapping

Programmatic Mapping

Mapping Operations

Importing Geospatial Data

Supported Geometry Types

Point Data Example

Complex Geometry Example

Geospatial Data Notes

Importing Blob Data

Base64 Encoding

File-Based Blobs

Pattern Token Reference

Mixing Strategies

Teradata Bulk Import: Staging Tables

How It Works

Controlling Clob and Geospatial Casting

Note

Choosing the right strategy for large data

Transaction Control

Note

Streaming Large Files

What Streaming Does Not Change

When to Use It

Importing GeoJSON

Warning

Nested Properties

Unmapped Columns

Streaming

Supported Documents

Note

Data Types

Limitations

Handling Autogenerated Columns (Identity)

Default Behavior (KeepIdentity = false)

Importing Identity Values (KeepIdentity = true)

Files With and Without Headers

With Headers (Default)

Without Headers

Positional by call order (string overload)

Positional by explicit index (integer overload)

Partial-column imports

Note

Handling Null Values

Null Value Examples

Import Code for Nullable Columns

All Data Types with Nulls

Field Delimiters and Special Characters

Custom Delimiters

Quoted Fields

Line Terminators

Encoding

Complete Examples

Example 1: Basic Import with Automatic Configuration

Example 2: Import with Custom Date Formats

Example 3: Import with Manual Column Mapping

Example 4: Import Geospatial Data with Identity Values

Example 5: Import File Without Headers

Example 6: Large File Import with Mixed Data Types

Example 7: Import with Tab Delimiter and European Format

Best Practices