FtpXml Connector - ItemUpdate

The ItemUpdate method in the FtpXml Connector is responsible for synchronizing item-related data from external XML files, primarily items.xml and itemfilters.xml, located on an FTP server. It updates App4Sales with item details, sales prices, VAT information, stock levels, item classifications, matrix/variant data, and embedded item images. The process involves downloading these XML files, parsing their content, applying various transformations and conditional logic, and then persisting the processed data into the App4Sales internal database.

Data Source Configuration

The connector retrieves data from the following sources, accessed via an FTP connection configured within App4Sales. File paths are relative to the configured FTP root directory.

items.xml: This is the primary data source for item details. It contains individual item records, their descriptions, pricing, stock information, EAN codes, matrix data, and embedded Base64 image data. This file is downloaded from the FTP server and processed.
itemfilters.xml: This XML file defines item classification categories. It is downloaded from the FTP server and parsed to establish a mapping between internal IDs and human-readable captions, which are then used to interpret item attributes from items.xml.
languages.xml: An optional XML file containing language codes. If present, it is downloaded and processed to update language descriptions within App4Sales. Errors during the processing of this file are logged but do not halt the item update process.
Internal App4Sales Database:
- VAT Code Percentages: Retrieved from VatCodePercentagesContext for VAT calculations.
- Stock Data: If configured to calculate the Stock Indicator by formula, base stock data for items is retrieved from ItemsContext.

Item Data Mapping

App4Sales Field	Source Field (API/Excel/DB)	Logic/Notes
`ItemCode`	`ID` (attribute of `<entityNode>`)	Directly mapped from the `ID` attribute of each item's XML node.
`Currency`	Hardcoded / `AdministrationSession`	If `AdministrationSession.CurrentSession.Administration.CurrencySymbol` is provided, its value is used. Otherwise, it defaults to "EUR".
`SalesPrice`	`items.salespackageprice`	Parsed from string to a decimal value. If parsing fails, the value will be `null`.
`EanCode`	`items.eancode`	Direct mapping. Values are HTML decoded unless they start with '<'.
`Sysmodified`	`items.sysmodified`	Parsed from string to a DateTime object.
`CreatedDate`	`items.createddate`	Parsed from string to a DateTime object.
`VatIncluded`	`items.vatincluded`	Parsed from string to a character. Items without a value in this field are skipped from processing.
`Description`	`items.description`	Direct mapping. Values are HTML decoded unless they start with '<'.
`PurchasePackageSize`	`items.purchasepackagesize`	Parsed from string to a decimal value.
`Description2`	`items.description2`	Direct mapping. Values are HTML decoded unless they start with '<'.
`Description3`	`items.description3`	Direct mapping. Values are HTML decoded unless they start with '<'.
`Description4`	`items.description4`	Direct mapping. Values are HTML decoded unless they start with '<'.
`Description5`	`items.description5`	Direct mapping. Values are HTML decoded unless they start with '<'.
`FreeSortField`	`items.freesortfield`	Parsed from string to an integer value.
`FreeItemFields`	`items.freeitemfields`	Direct mapping. Values are HTML decoded unless they start with '<'.
`SearchDescription`	`items.searchdescription`	Direct mapping. Values are HTML decoded unless they start with '<'.
`Unit`	`items.unit`	Direct mapping. Values are HTML decoded unless they start with '<'.
`VatPercentage`	`items.vatpercentage`	If the source contains '`~`' (e.g., "CODE~COUNTRY"), it's split. A lookup is performed against predefined VAT percentages. If a match is found for the country and code, that percentage is used. Otherwise, it defaults to 0, and a warning is logged. If no '`~`' is present, the value is parsed directly to a decimal.
`LastAvailableStock`	`items.lastavailablestock`	Parsed from string to a decimal value.
`NextDelivery`	`items.nextdelivery`	Parsed from string to a nullable DateTime object.
`MatrixParent`	`items.matrixparent`	Direct mapping. Used to identify the parent item of a variant. Values are HTML decoded unless they start with '<'.
`MatrixXDescription`	`items.matrixxdescription`	Direct mapping. Describes the X-axis of a matrix (e.g., "Color"). Values are HTML decoded unless they start with '<'.
`MatrixXValue`	`items.matrixxcode`	Direct mapping. The value for the X-axis of a matrix (e.g., "Red"). Values are HTML decoded unless they start with '<'.
`MatrixYDescription`	`items.matrixydescription`	Direct mapping. Describes the Y-axis of a matrix (e.g., "Size"). Values are HTML decoded unless they start with '<'.
`MatrixYValue`	`items.matrixycode`	Direct mapping. The value for the Y-axis of a matrix (e.g., "Large"). Values are HTML decoded unless they start with '<'.
`RelatedItemsText`	`items.relateditemstext`	Direct mapping for a search field used to link items on the scan page. Values are HTML decoded unless they start with '<'.
`RelatedItems`	`items.relateditems`	Direct mapping for XML content containing linked items on the item detail card. Values are HTML decoded unless they start with '<'.
`MinimumOrderQuantity`	`items.minimumorderquantity`	Parsed from string to a decimal value.
`StockIndicator`	`items.stockindicator` / Derived	If the 'Use stock indicator from connector' setting is true, maps directly from `items.stockindicator` (parsed to integer). Otherwise, if the 'Get stock indicator by formula' setting is true, it is derived using an internal formula based on available stock and administrator-defined thresholds (see "Stock & Availability").

Item Classification Mapping

Item classifications (ItemClasses and their ItemClassValues) are derived from two sources: itemfilters.xml and specific fields within items.xml.

App4Sales Field	Source Field (API/Excel/DB)	Logic/Notes
`ItemClass.Description`	`Caption` attribute from `itemfilters.xml`	The `itemfilters.xml` file contains entries like `<class ID="1" Caption="Color"/>`. The `Caption` attribute (e.g., "Color") becomes the `ItemClass.Description`.
`ItemClassValue.Value`	`itemclassvalues.valueX` from `items.xml`	The value of fields such as `itemclassvalues.value1`, `itemclassvalues.value2`, etc., is mapped to the `Value` of an `ItemClassValue`. The numeric suffix 'X' (e.g., '1') is used to look up the corresponding `ItemClass.Description` from `itemfilters.xml`. Values are HTML decoded unless they start with '<'.

Price List Item Mapping

Additional sales prices beyond the primary SalesPrice are mapped to PriceListPrice entities, creating dynamic Price Lists as needed.

App4Sales Field	Source Field (API/Excel/DB)	Logic/Notes
`PriceList.Code`	Derived from `items.price_X`	The numeric suffix 'X' (e.g., '2' from `items.price_2`) is used as the Price List Code.
`PriceList.Description`	Derived from `items.price_X`	Automatically generated as "Pricelist X" (e.g., "Pricelist 2").
`PriceList.Id`	Derived from `items.price_X`	The numeric suffix 'X' is also used as the Price List ID.
`PriceListPrice.PriceListId`	Derived from `items.price_X`	The numeric suffix 'X' of the source field.
`PriceListPrice.ItemCode`	`Item.ItemCode`	The App4Sales `ItemCode` of the item.
`PriceListPrice.Price`	`items.price_X` (value)	The value of the `items.price_X` field, parsed from string to a decimal. Only valid decimal values are processed.

Special Logic & Filters

Item Filtering by VAT Included: Any item record in items.xml that does not contain a value for items.vatincluded will be skipped entirely.
Item Filtering by Price List Presence: The system can be configured via the 'Only import items with pricelist' setting. If this setting is active (set to 1, 2, 3, or 4), items will only be imported if the corresponding items.salespackageprice, items.price_2, items.price_3, or items.price_4 field, respectively, contains a value. If the required price field is missing, the item is skipped.
Language Synchronization: The SyncLanguages() helper method attempts to download and process languages.xml. This file is expected to contain language codes which are used to update App4Sales language descriptions. Any errors during the download or parsing of languages.xml are logged but do not prevent the main item update process from continuing.
Error Handling: The process includes robust error handling with try-catch blocks around XML file downloads and item processing loops. Errors are logged, and in critical cases, Session.PortalServerProvider.ResetDataUpdateIntervals() is called to manage update intervals.

Domain Specifics (Expanded)

Price Logic

The connector supports a base sales price and multiple additional price lists, along with detailed VAT handling.

Base Sales Price: The core selling price is mapped from items.salespackageprice to Item.SalesPrice.
Tiered/Customer-Specific Prices: Additional price lists are dynamically created and populated from fields like items.price_2, items.price_3, etc. Each items.price_X field (where 'X' is a number) represents a specific price list. If a value is present, a PriceListPrice record is created for the item within that price list.
VAT Calculation:
- Item.VatIncluded indicates if the sales price already includes VAT.
- Item.VatPercentage is determined based on the items.vatpercentage field. This field can either contain a direct percentage or a combined code and country (e.g., "HIGH~NL") which is then looked up against pre-configured VAT percentages in App4Sales. If a combined code/country is used and no match is found, the VAT percentage defaults to 0.

Image Handling

Item images are handled directly through embedded Base64 strings within the items.xml.

Image Source: Images are expected to be provided as Base64 encoded strings in the items.picture field of items.xml.
Processing: The Base64 string is decoded into a byte array and associated with the item using a method that includes a hash check, likely to prevent duplicate image storage.
Optimized Updates: If the 'Only update new pictures' setting is enabled, the 'lastupdated' timestamp for existing item pictures in the database is updated at the beginning of the sync, potentially optimizing subsequent image processing.
Cleanup: A temporary directory used for item pictures, determined by FtpSyncManager.Instance.ItemPicturesFolder, is deleted after the update process completes.

Stock & Availability

Stock information primarily consists of available stock, next delivery dates, and a stock indicator.

Available Stock: The items.lastavailablestock field is mapped to Item.LastAvailableStock, representing the quantity of stock currently available.
Next Delivery: The items.nextdelivery field is mapped to Item.NextDelivery, providing an estimated date for future stock availability.
Stock Indicator: The Item.StockIndicator can be sourced directly from items.stockindicator in the XML if the 'Use stock indicator from connector' setting is true. Alternatively, if the 'Get stock indicator by formula' setting is true, the indicator is dynamically calculated. This calculation involves fetching additional stock data from App4Sales' internal database and using an external script (CalculateStockIndicatorByFormula) that considers thresholds (QuantityForRed, QuantityForGreen from App4Sales settings) to determine the indicator status (e.g., green, orange, red).

Matrix & Attributes

The connector supports robust handling of matrix items (parent-child variants) and custom item attributes.

Matrix Items:
- Parent-Child Linking: The items.matrixparent field explicitly links a child variant to its parent item.
- Variant Axes: Fields like items.matrixxdescription/items.matrixxcode and items.matrixydescription/items.matrixycode define the characteristics (e.g., "Color", "Size") and their respective values for each variant.
Item Classifications (Attributes):
- Custom attributes are defined using itemfilters.xml (which maps internal IDs to descriptions like "Material" or "Brand").
- Specific attribute values for each item are provided in items.xml using fields like itemclassvalues.value1, itemclassvalues.value2, etc. These values are then associated with their corresponding item class descriptions.

Related Settings & Prerequisites

The behavior of the Item Update process can be influenced by several connector and App4Sales settings:

Only update new pictures: (Connector Setting) If true, a database update timestamp for item pictures is set, potentially influencing how images are re-processed.
Only import items with pricelist: (Connector Setting) An integer value (1-4) that determines which sales price field (items.salespackageprice, items.price_2, items.price_3, or items.price_4, respectively) must have a value for an item to be imported.
Use stock indicator from connector: (Connector Base Setting) If true, the Item.StockIndicator is taken directly from the items.stockindicator field in the XML.
Get stock indicator by formula: (Connector Base Setting) If true (and 'Use stock indicator from connector' is false), the Item.StockIndicator is calculated using an internal formula and script.
Quantity for Red: (App4Sales Setting) A threshold used in the stock indicator formula to determine when stock should be flagged as 'red'.
Quantity for Green: (App4Sales Setting) A threshold used in the stock indicator formula to determine when stock should be flagged as 'green'.
XML File Structure: The connector expects items.xml, itemfilters.xml, and optionally languages.xml to adhere to a specific XML structure for successful parsing.
FTP Connection: A correctly configured FTP connection (host, credentials, root directory) is essential for the connector to download the necessary XML files.

Known Limitations

languages.xml Error Handling: Errors encountered during the download or parsing of languages.xml are logged but are intentionally suppressed, meaning language synchronization failures will not prevent the primary item update from completing. The comment in the code suggests this is a low-priority process.
Stock Indicator Formula Complexity: The exact logic for the 'Stock Indicator' when calculated by formula is defined in an external script (DataConstants.Scripts.CalculateStockIndicatorByFormula). Administrators configuring the system would need to understand this script's logic to predict stock indicator behavior accurately. The C# code only orchestrates the execution of this script.
Image File Naming/Structure: While Base64 embedded images are supported, the documentation from code analysis does not indicate support for image files on the FTP server with specific naming conventions for linking to items. The primary image handling observed is via Base64.

FtpXml - Item sync