Stream Grids 4.what are messages

Messages are used to define the format and schema(fields and their datatypes) of the data that will be received from the streams. Each message created by the user will be tied to a Source configuration and the schema associated with this message can evolve as the message passes through various transformations.

Message Name: A unique qualifier for the message

Use Default Message: Available options yes/no. If option Yes is selected, user can choose from templates like Apache Logs, Router Logs, Twitter tweets etc. Otherwise, user has to provide the message details in next step.

Message Format: Supported Formats are JSON, XML, Delimited, Regex.

Persistence Id: Acts as a unique id (Row key in HBase) for each record while persisting to a persistent store. User can accept a UUID implementation or can provide a fully qualified Class name which generates the ID.

Index Id: This field acts as the unique identifier for the record in the index store. Similar to above, user can choose a UUID/custom implementation.

For message formats like JSON/XML user would be required to upload a sample file and the schema would be automatically inferred.