Skip to main content

DataImport Schema

Top-level Message Structure​

DataImport Event uses the following schema.

Top level message structure
{
    "eventType": "externaldata",
    "eventSubtype": "dataImport",
    "correlationId": "{{$guid}}",
    "source": "MDM",
    "priority": 2,  //Optional. Allows calling system to direct incoming event to a specific FIFO queue for
                    // processing. Currently there are two queues: Bulk (1 or less) and Normal (2 or more).
                    // If not specified default priority will be assigned based on
                    // # of entities < 20 and total message size < 40kb.
    "payload":
    {
        "EventIdentifier": "CRM-Onboarding-Event", //Identifies event configuration (visible in Integration Hub)
        "IntegrationName": "CRM Onboarding",  //Identifies overall integration process. Included in any logs produced
        "IntegrationStatus": "Success", //Optional. Allows client to provide a custom status (e.g. in case of upstream failures)

        //Entities and other Domain Data to be updated in Fenergo
        "Entities": [], //Entities to be created/update or referenced by other domains
        "EntityAssociations": [], //Optional. Entity Association Hierarchies
        "GroupAssociations": [], //Optional. Future Capability
        "Products": [],  //Optional. Products to be created/updated

        "EventResponseParams": {
            //Used in CLM-inititated Events where the incoming event is updating a journey already in progress.
            //Provides details for the Entity and Journey in progress and bypasses the JourneyInitiation Process
            //Errors will be aggregated up to the Root Entity/Journey
            "RootEntity": "", //EntityId of the journey in progress
            "JourneyId": "", //JourneyId that is expecting the incoming event
            "TaskId": "" //Blocking Task to be completed once the event is processed
        },

        "ExternalLogs": [ ], //Optional. Upstream integrations can provide error details to be viewed in Fenergo
        "EventConfig": { } //Event Configurations (Data Mapping and Behaviours) are configured here
    }
}

Entities​

Entities
    "Entities": [
    {
        /***
         * Entities can be resolved by either
         *     -their FenX Entity Id if known by the calling system,
         *     - or using a unique system Id via EntityResolver
         * Or force a new entity to always be created by providing an empty GUID ({0000-000...}) as Entity.Id
         ***/
        "id": "",
        "entityResolver": {
            /**
            Entity Resolver identifies an Entity using an _unique_ external identifier (externalSystemId) stored in a datakey (externalDataKey) that
            has been configured as 'indexable' in the policy. An example would be MDMID managed by a Client's central MDM
            **/
            "externalDatakey": "{{Datakey}}",
            "externalSystemId": "{{Value}}"
        },
        "journey": {
            //Optional. When updating multiple inflight journeys / entities you can specify the running journey Id and optional task to be closed.
            "journeyId": guid,
            "taskId": guid,
            "taskKey": string
        }
        "entityType": "Company", //Optional. Entity Type is only required if Entity does not exist and must be created
        "targetEntity": "Client", //Optional. If the entity is known to be a Client or RelatedParty by the external system then this can be specified in the message and used to determine what journey to trigger via JourneyLogic
        "lifecycleStatus": "OnboardingInProgress", //Optional. If the entity lifecycle is known by the external system then this can be specified in the message. See JourneyMetadata.Lifecyclestatus for possible values.
        "jurisdictions": [ "Global" ], //Provide a list of jurisdictions to be offboarded / reonboarded for given entity
        "singleProperties": {
            //Collection of datakey names and values.
            //DataImport uses Policy config identified in EventConfig to validate data types and values. Lookup fields can map by Value or ValueId
            "legalEntityName": "LexCorp Industries",
            "countryOfJurisdiction": "Canada", //Lookup by Value
            "countryOfIncorporation": "CA", //Lookup by ValueId
            "dateOfIncorporation": "2012-07-10", //Date
            "multiSelectField": "CA|IE|FR", //Multiselect lookup with three values
        },
        "collectionProperties":
            //Contains a collection of DataGroups
            {
            "addresses": { //DataGroup Name
                "mergeStrategy": "replace",  //Replaces any existing datagroup records with those received in this event
                "records": [{
                   "type": "Registered",
                   "addressLine1": "123 Main ST",
                   "state": "NY",
                   "country": "US"
               }]
            }}
    }, //{ Entity #2}
    ]

EntityAssociations​

Entities
"EntityAssociations": [
    //Supports Entity Associations (e.g. {source} is a Director/Shareholder of {target} )
    {
        //Either EntityId / EntityExternalId required and must map to an Entity within this message.
        //Associations will be created with the entity journey
        "EntityId": "",
        "EntityExternalId":"", //maps to Entity.EntityResolver.ExternalSystemId
        "ReplaceAssociations": boolean, //Deletes any associations in Fenergo that aren't provided in this message
        "Associations": [
            {
                //Either TargetId (resolves to FenX Entity Id)
                //or TargetExternalId (Resolves to a matching Entity provided in this message using Entity.EntityResolver.ExternalSystemId)
                "TargetId": "",
                "TargetExternalId": "",

                "SourceId": "",
                "SourceExternalId": "",

                "AssociationType": "Parent", //AssociationType must be a value from Relationships lookup list
                "OwnershipPercentage": null, //Optional
                "ControlPercentage": null //Optional,
                "SingleProperties": {           // Custom association properties
                    "datakey1": "value",
                    "datakey2": "value"
                }
            } //, { .. }
]}]

Products​

Entities
"Products": [
    {
    /***
    Product Resolving:

    Id - Fenergo Product ID
    ProductResolver - DataImport will search for existing products linked to the Entity that have a unique field <externalDatakey> == <externalSystemId>

    Either ID or ProductResolver must be specified
    ***/

    //"Id": "864c7d39-926d-4ee4-a9c5-98e24efc1ac9",
    "ProductResolver": {
        "externalDatakey": string, //Product datakey Id
        "externalSystemId": string //Datakey value
    },

    /****
    Product Owner Entity must be included in the message (Entities). Can be resolved within the event using EntityId or EntityReferenceId which searches Entities for matching Entity.EntityResolver.ExternalSystemId

    Either EntityId or EntityReferenceId must be specified
    **/
    "EntityId": string, //guid
    "EntityReferenceId": string,

    //Type & Family are required when creating new Product, otherwise optional
    "Type": string,
    "Family": string,
    /**
    Status - [ InProgress | Offboard | Reonboard ]. Optional field, defaults to "InProgress"

    Offboard
         - If Product has never been verified it will be deleted permanently.
         - For verified Products DataImport will delete product draft (if one exists) and mark Product to be offboarded at end of Journey.
    Reonboard - Removes offboarded flag on Product
    ***/
    "Status": string,
    /**
     * SingleProperties and CollectionProperties works the same as on Entities
     *
     * Select or MultiSelect fields can provide Reference List ID or Value.
     */
    "singleProperties": {
        "datakey1": string,
        "datakey2": number
        //...
    },
    "collectionProperties": {
        "bookingDetails": {
            "MergeStrategy": "replace",
            "Records": [
                {
                    "datakey1": string,
                    "datakey2": number
                    //...
                }
            ]
        }
    }
    }//, { .. }
]}]

EventConfig​

EventConfig
"EventConfig": {
        "JourneyLogic": [
            /** JourneyLogic mapping determines which journey to trigger when processing incoming entity data changes.
            JourneyLogic now supports the following factors when determining which journey to trigger: EntityType, ExistingEntity, targetEntity (e.g. Client), and lifecycleStatus (e.g. Compliant or Onboarding in Progress). Factors are optional if not applicable. **/
                {
                    "entityType": "Company",
                    "existingEntity": true,
                    "targetEntity": "Client",
                    "lifecycleStatuses": [ "Compliant", "Onboarding in Progress" ],
                    "journeySchemaIdentifier": "maintenance-journey-1",
                    "taskDataKey": "ingress_blocking_task"
                }//, {..}
        ],
        "EntityProcessingRules": {
            //Policy Mapping
            "entityCreateJurisdictions": ["Global"], //Optional, defaults to "Global". Initial Jurisdictions when creating Entity
            "offboardingJurisdictions": ["Global"], //Optional. Applies Jurisdictions to all Entities where Entity.Jurisdiction not specified and Journey Type is "Client Offboarding"
            "reonboardingJurisdictions": ["Global"], //Optional. Applies Jurisdictions to all Entities where Entity.Jurisdiction not specified and Journey Type is "Client ReOnboarding"
            "entityCreateCategories": ["Basic Details (MDM)"],
            "entityUpdateCategories": ["Basic Details", "Basic Details (MDM)", "Onboarding Event Review"],

            "ImportErrorsCategory": "Onboarding Event Review", //Optional
            "importStatusDataKey": "externalEventStatus", //Optional
            "ImportErrorsDataGroup": "errorDetails", //Optional
            //note: importStatusDataKey and ImportErrorsDataGroup needs to be exposed in both entityUpdateCategories and ImportErrorsCategories

            //Behaviours
            "ErrorOnCreateEntity": false, //Event should only update existing records. Throw exception if existing record not found
            "IgnoreEntityFieldsOnUpdate": false,  //Use incoming entity data to create entities only, never override existing data in Fenergo. Useful when integrating with systems with untrusted data (e.g. CRM Product Onboarding Requests)
            "NewEntityJourneyForNonClients": false //Trigger the COB for existing non-client entity
            "waitForSearchPropagation": true  //Optional. Verifies newly created entities have finished updating
                                              // Entity Search datastore before completing processing.
        },
        "ProductProcessingRules": {
            //Policy Mapping
            "ProductJurisdictions": ["Global"], //Optional, defaults to "Global". Initial Jurisdictions when creating Product
            "ProductCategories": ["Product Details"], //Product Requirement Set Categories (Policy) to derrive data types and validations during Product Update"
        },
        "ImportBehaviors": {
            //Any special rules related to business logic while processing the import message go here
            "SkipCompleteTask": false, //Do not attempt to close journey task at end of processing
            "DelayedCompleteTask": false //Journey Task may not be in "ready" state at the start of processing. Skip this validation step
        }  }

ExternalLogs​

Incoming ExternalLogs are merged with errors and warnings raised while processing the event. These Log entries all follow the same schema below. They are logged to LoggingCentre and visible in Integration Hub.

You can optionally merge log entries to the Entity Draft if you have specified importErrorsDataGroup in EventConfig.EntityProcessingRules. The datakeys you configure in that datagroup must use the same name for the data to be merged. You can also omit fields from being merged by not defining a field in your datagroup.

The Event will be marked as a Failure if ImportStatus is not "Success" or if there are any imported Entities with log entries with an Error Severity.

ExternalLogs
"externalLogs": [
    {
        "severity": "Information", // Information | Warning | Error
        "integration": "MDMImport",
        "sourceSystem": "MDM",
        "description": "Failure while fetching MDM record",
        "details": "MDM Id: 123456. MDM System Unavailable",
        "errorCode": "ClientErrorCode"
    } //,{}...
]

Sample Messages​

  • External Event: This message demonstrates an External Event integration pattern. It simulates an external system sending an Entity Update or New Entity to be onboarded. If the entity exists and is a customer it triggers a Maintenance Journey, else it triggers a Client Onboarding journey.
  • CLM-inititated Event: This message demonstrates the CLM-inititated Event integration pattern. It This message updates updates that exists in a customer's MDM system, and includes any existing associations.
  • Offboarding Event: This message demonstrates using a DataImport event to trigger an offboarding journey to offboard the Canadian Jurisdiction of an Entity.