Offline Data Feed

Offline data is merchant information that is not directly collected by Algonomy. Some examples include in-store (POS) transactions, call center transactions, marketing segments, other marketing data, and social media interactions.

Offline data powers some key elements of Algonomy omnichannel personalization:

Cross-channel attribution: Algonomy algorithms use a closed loop optimization cycle to determine what’s working. A recommendation or online session that is finished in an offline transaction should be considered a success, not a failure. Siloed data – this is when we don’t take offline transactions into account – may suggest failure. As an additional bonus, we provide omnichannel attribution reporting to you.
Omnichannel data for smarter analysis: When targeting products and making offers to your shoppers, you should consider as much data as possible. Offline data is important when selecting appropriate upsells and cross-sells. Our algorithms can better predict behavior when looking at all trends of your shoppers’ behaviors.

Benefits of offline data analysis

Immediate: Shoppers who buy offline and are recognized online receive more relevant recommendations on home and category pages. The pages are tailored for cross-sells. (Conversion)
Immediate: Price, brand, category, size, and style preferences for recognized online shoppers become more comprehensive. (Conversion, AOV)
Mid-term: Algonomy predictive models that use omnichannel data are smarter by having more shopper information.
Long-term: Personalization and loyalty are tied together.

Support for Importing Offline Data through APIs

Algonomy Omnichannel Personalization allows you to import offline data through APIs. For more information see: Omnichannel Order API.

Validating and Uploading the Feed

Before scheduling the feed for regular uploading, generate a sample feed. Validate the feed to confirm that the structure is correct. After the feed is validated, send the sample to the Algonomy integration team for processing. Once the test file has been successfully processed, the production file should be scheduled for upload on an agreed schedule, typically once per day in the early morning (between 12am and 4am).

All files for the feed will be uploaded as a single compressed file (.zip or .gz). Files should be sent to the Algonomy FTP server (your integration team will provide you with your FTP credentials).

Bundling Multiple Files

Multiple files are to be delivered in a single .zip file. In this case, name the bundle with a consistent prefix and the _SITENAMEONEWORD_SITEID_YYYY_MM_DD_vNUM postfix, but the files within should retain the standard names above.

For example, a zip file with the name offline_orders_myshop_123_2013_01_12_v1.zip may contain the files

orders_myshop_123_2013_01_12_v1.txt
order_line_items_myshop_123_2013_01_12_v1.txt
returned_orders_myshop_123_2013_01_12_v1.txt
returned_line_items_myshop_123_2013_01_12_v1.txt

Coordinate with your Algonomy team to choose the prefix you’ll use.

Standard Naming Convention and Formats by File Type

You will receive a site ID for your site(s) during the setup process. Each filename should include both your site name and your site ID.*

Additionally, if you plan to upload multiple zip files each day, it is a good idea to use versioning in the filenames. For feed types which overwrite data (e.g., store data), the version signals to the system that we’re progressing. For feed types which can be appended incrementally through the day (e.g., orders where the first file is for purchases from midnight-11 AM, and the second is from 11 AM to midnight) the versions signal incrementally. If a version goes backwards, the system will archive the file and flag an error.

For Example:

MyShop.com is assigned SiteID 123
MyShop desires to upload an order feed
The order feed uses the filename format: orders_SITENAMEONEWORD_SITEID_YYYY_MM_DD_vNUM.zip

Compressed* feed file: purchases_myshop_123_2013_01_12_v1.zip

Inside of the compressed file: purchases_myshop_123_2013_01_12_v1.txt

Note: The file can be compressed as .zip or .gz, with the extension matching the type of compression.

Below are the standard feed file names for each standard type. Every file should end with the short site name, Site ID, Date, and version fragment, e.g. for MyShop.com, site 123 and January 12, 2013: _myshop_123_2013_01_12_v1.txt

The version fragment (‘v1′ above) can be a timestamp or other increasing value instead of a number. The important part of the version is distinguishing the order of arrival of multiple feeds in a single day.

Note: The 'site name' is technically your ftp login name. If there has been any changes to your ftp login name, please use that instead of "site name".

Feed Files

Feed File	Details	Filename pattern
Order Feed	A list and description of every order made over the decided time period.	orders_sitename_siteID_YYYY_MM_DD_v#.txt
Order Line Item Feed	Line item details for each order in the Order Feed.	order_line_items_sitename_siteID_YYYY_MM_DD_v#.txt
Returned Order Feed	A list and description of every return made over the decided time period.	returned_orders_sitename_siteID_YYYY_MM_DD_v#.txt
Returned Line Item Feed	Line item details for each return in the Returned Order Feed.	returned_line_items_sitename_siteID_YYYY_MM_DD_v#.txt
Shopper GUID Mapping Feed	A mapping between disparate shopper guids to a central, unifying shopper id that you specify.	shopper_guid_mapping_sitename_siteID_YYYY_MM_DD_v#.txt

Order and Return Feeds

Orders and returns are separate feeds with different data. Offline purchases are broken down into orders, individual receipts, or transactions.

Order data consists of the purchase date, shopper information, and the total cost of all products for that transaction.

Orders are line items that represent different products purchased by the shopper. Think of it as an itemized receipt: the order is the receipt, and a line item is one product.

Order Feed: this describes entire orders.
Line Item Feed: this describes individual line items.

Order Feed

Filename: orders_sitename_siteID_YYYY_MM_DD_v#.txt required

Name	Type	Required?	Definition
order_id	ASCII	Yes	Numeric value for each order. At least unique by channel.
channel_id	ENUM	DEPENDS	The identifier for the channel of this shopper mapping, e.g. “Store.” If you are mapping website shopper guids to shopper IDs, the channel id should be provided that matches the channel id in the Shopper Guid mapping file.
shopper_guid	text	DEPENDS Required only if shopper_id is not provided.	Shopper guid that maps to a shopper id in a given channel. Refer Shopper GUID Mapping Feeds, below. This only required if shopper id is not provided. IMPORTANT: A separate user mapping feed must be provided. Note: Either shopper_guid or shopper_id are required. If both are blank, the row will be ignored. If neither can be provided, but the retailer still wants the data included, a unique arbitrary ID can be provided.
shopper_id	Text	Depends Required only if shopper_guid is not provided.	Global Unique Shopper ID. Shopper IDs in the offline feed must match shopper IDs received through site instrumentation. Note: Either shopper_guid or shopper_id are required. If both are blank, the row will be ignored. If neither can be provided, but the retailer still wants the data included, a unique arbitrary ID can be provided.
date_time	ISO DATETIME	Yes	A date and time of the transaction in ISO 8601 compatible format. Example: 2007-04-25T14:30:00+09:00 Note: The date_time format should match the ISO 8601 compatible format.
email_hash	Text	No	A hash of the customer’s email address, if POS captures email at transaction time. Hashed out for PII standards.
cc_hash	Text	No	A fully obfuscated credit card identifier (See Data Obfuscation discussion above).
payment_method	ENUM	No	e.g. cash, credit, check, digital, other. Useful for segmentation and targeting
store_id	Text	No	A store in which the transaction occurred.
region_id	Alphanumeric	No	The region_id in which the transaction occurred, as specified in the Region Full Feed
segment_id	Text	No	A unique identifier for the segment the shopper belongs to. A segment is a group of shoppers with a common demographic or demonstrated pattern to which promotions can be targeted based on affinities and patterns specific to that group. Note: Multiple segments can be sent in this field. The standard delimiter is a comma.
segment_name	Text	No	A human-friendly name for the segment. Note: Multiple segment names can be sent in this field. The standard delimiter is a comma.
currency	ENUM	No	The ISO currency code representing the currency used for the order. Note: If not specified, this defaults to US currency (USD). Don't add the currency column in the header if you don't want to send the appropriate ISO currency code against each row. By default, system sets the currency to USD. If you add the currency column in the header, the default currency won't be applied.
coupon_code	Text	No	Was a coupon applied to the entire order? If so, you can record the coupon code here.
coupon_value	NUMBER	No	Monetary value of the coupon applied to the entire order.

Line Item Feed

Filename: order_line_items_sitename_siteID_YYYY_MM_DD_v#.txt required

Name	Type	Required?	Definition
order_id	ASCII	Yes	As defined above. Required to associate the line item data with the order data above.
channel_id	ENUM	No	The identifier for the channel of this shopper mapping, e.g. “Store”. If you are mapping website shopper IDs to shopper GUIDs, then the value of the channel ID is determined by the “Omnichannel Web Channel” site configuration value.
product_id	text	Yes	Unique ID for a product. This should be a key to look up into the catalog.
sku_id	text	No	SKU for the line item. Should correspond to a value provided in the SKU Feed
quantity	INTEGER	Yes	Quantity of the items purchased. This value needs to be an integer.
unit_price	NUMBER	Yes	Monetary value paid per unit
coupon_code	text	No	Was a coupon code used for this individual purchase/line? If so, you can record the coupon code here.
coupon_value	NUMBER	No	Monetary value of the coupon applied to this individual line item.

Returned Order Feed

Filename: returned_orders_sitename_siteID_YYYY_MM_DD_v#.txt

Field Name	Type	Required?	Definition
order_id	ASCII	Yes	Numeric value for each order. At least unique by channel.
return_id	text	No	Unique return ID.
channel_id	ENUM	DEPENDS	The identifier for the channel of this shopper mapping, e.g. “Store.” If you are mapping website shopper guids to shopper IDs, the channel id should be provided that matches the channel id in the Shopper Guid mapping file.
shopper_guid	Text	DEPENDS Yes, but only if shopper_id is not provided IMPORTANT: A separate user mapping feed must be provided.	Shopper GUID as defined above in the orders feed.
shopper_id	Text	DEPENDS	As defined above in the orders feed.
date_time	ISO DATETIME	Yes	A date and time of the transaction in ISO 8601 compatible format. Example: 2007-04-25T14:30:00+09:00
email_hash	Text	No	A hash of the customer’s email address, if POS captures email at transaction time.
store_id	Text	Yes	A store in which the transaction occurred
segment_id	Text	No	A unique identifier for the segment the shopper belongs to. A segment is a group of shoppers with a common demographic or demonstrated pattern to which promotions can be targeted based on affinities and patterns specific to that group.
segment_name	Text	No	A human-friendly name for the segment.
return_value	NUMBER	Yes	Monetary value of the return.
currency	ENUM	No	The ISO currency code representing the currency used for return_value. Note: If not specified, this defaults to US currency (USD). Don't add the currency column in the header if you don't want to send the appropriate ISO currency code against each row. By default, system sets the currency to USD. If you add the currency column in the header, the default currency won't be applied.

Returned Line Item Feed

Filename: returned_line_items_sitename_siteID_YYYY_MM_DD_v#.txt

Name	Type	Required?	Definition
order_id	ASCII	Yes	Numeric value for each order. At least unique by channel.
return_id	text	No	Unique Return ID.
channel_id	ENUM	Yes	The identifier for the channel of this shopper mapping, e.g. “Store”. If you are mapping website shopper IDs to shopper GUIDs, then the value of the channel ID is determined by the “Omnichannel Web Channel” site configuration value.
product_id	text	Yes	Unique ID for a product. This should be a key to look up into the catalog.
sku_id	text	No	SKU for the line item. Should correspond to a value provided in the SKU Feed.
quantity	INTEGER	Yes	Quantity of the item(s) returned. This value needs to be an integer.
unit_value	NUMBER	Yes	Monetary value refunded per unit returned.
credit_type	ENUM	Yes	Did this result in chargeback, cash back, store credit, etc.

Orders and Returns Events

Orders and returns in the offline world are events that occur at a specific time and place, and can be tracked and ingested in real-time, or as part of a feed. Unlike the feeds below, orders and returns are things that have happened, rather than descriptions of things that exist, such as products, shoppers, or stores.

At Algonomy, we refer to data about things that have happened event data and have a globally distributed event ingestion system for online events. We call data about things that exist as entity data. Currently, we accept up to 14 days' worth of Omnichannel event data daily through batch data feeds. In the future, we anticipate that Omnichannel event data will be consumed via real-time data ingestion instead of batch data feeds.

Note: The platform will process only offline orders and returns data that was created within the past 14 days from the import date.

Shopper GUID Mapping Feed

Filename: shopper_guid_mapping_sitename_siteID_YYYY_MM_DD_v#.txt

If the merchant does not include shopper IDs in the offline feed, the transaction information will be used to create the global omnichannel strategies. The feed needs to have shopper IDs for mapping to be able to present personalized omnichannel strategies.

Ideally, a merchant would have a globally-unique shopper identifier that functions across channels. Realistically, merchants have some systems and channels operating with such a shared identifier, and other systems and channels with their shopper identifiers.

A single shopper GUID mapping feed can map shopper GUIDs from various channels to the Shopper ID, where this mapping is known. This is the simplest way to connect users across channel when possible.

This mapping feed can map disparate shopper GUIDs to a single, known shopper ID. Each line in this file maps to one shopper ID, and multiple lines can be used to map multiple shopper GUIIDs to the same shopper ID.

Name	Type	Required?	Definition
shopper_guid	ASCII	Yes	The per channel shopper GUID.
channel_id	ENUM	Yes	The identifier for the channel of this shopper mapping, e.g. “Store”.
shopper_id	text	Yes	The globally unique shopper ID.

Offline Feed Format, Packaging, and Delivery

A single feed is represented by a delimited text file in UTF-8 encoding without BOM (byte order mark). The file contains a single header row with the header labels for each column. Duplicate column names are not allowed, and data is matched to columns by name. The required fields must be included. Fields may be given in any order and unused ‘optional’ fields simply omitted.

The file has rows delimited by the ASCII newline character, and columns delimited by the ASCII pipe (|) character, which is not customizable. The only supported column delimiter is the pipe character (|). By default, there is no quoting or escaping of values, so any fields that contain newline or pipe characters must first remove or substitute these characters.

For example, the first two lines of an Order Feed might look like:

Copy

order_id|shopper_guid|channel_id|store_id|date_time|coupon_code|shopper_id
abc123xyz|987654321ab|store|s555-333|234.99|2007-04-25T14:30:00Z|jan_promo5|null

One-Time Historical Transaction Upload

Clients can provide a one-time historical transaction upload with more than 14 days of transactions. The only difference: the one-time filename starts with “historical_”. If you have already sent daily feeds, this one-time file must exclude transactions already delivered. They will not be de-duplicated.

IMPORTANT: The one-time historical transaction upload runs once weekly on Sunday.

Example filename: historical_offline_orders_myshop_123_2013_01_12_v1.zip.

The historical order feed should use the filename format: historical_offline_orders_SITENAMEONEWORD_SITEID_YYYY_MM_DD_vNUM.zip

Upload this file to the same FTP directory. Everything else remains the same.

Note: The sub files do not require the “historical_” prefix.

orders_myshop_123_2013_01_12_v1.txt
order_line_items_myshop_123_2013_01_12_v1.txt
returned_orders_myshop_123_2013_01_12_v1.txt
returned_line_items_myshop_123_2013_01_12_v1.txt