MongoDB extended JSON

Since all MongoDB queries from within Insights (UI and REST API) require to be written with extended JSON, this chapter provides an introduction to it and shows an example usage. This leads directly to the question of the necessity of an extended JSON format: Since raw JSON was not invented for storing documents with various data types (but to serialize data with less overhead than XML), MongoDB stores documents using BSON, the binary version of JSON, giving it the possibility to preserve data types. Under the hood, this makes sorting, comparing and indexing of documents way more efficient. Further on, only strict mode is supported by Bosch IoT Insights, meaning that any JSON parser would still parse strict mode representations of key/value pairs but only the internal parser of MongoDB recognizes actual type information.

Bosch IoT Insights MongoDB queries therefore need to comply with the extended JSON format. This requires to transcribe usual MongoDB queries, which other clients (like MongoDB Shell for instance) could handle. For most Insights queries, mainly two data types are relevant:

Date and its associated representation { "$date": "<date>" }
ObjectId and its associated representation { "$oid": "<id>" }

The following examples show how queries need to be adapted to comply with the extended JSON format. Doing so, exemplifications of the two essential data types (Date and ObjectId) are given below.

Conventional MongoDB query using the Date type:

{
    $match:{
        metaData.timestamp:{
            $lte:ISODate('2017-10-01T15:00:00.000Z')
        }
    }
}

Adapted MongoDB query using the Date type (compliant to extended JSON):

{
    "$match":{
        "metaData.timestamp":{
            "$lte":{
                "$date":"2017-10-01T15:00:00.000Z"
            }
        }
    }
}

Conventional MongoDB query using the ObjectId type:

{
    $match:{
        _id:ObjectId("5955f9618c9cdc0010e618f0")
    }
}

Adapted MongoDB query using the ObjectId type (compliant to extended JSON):

{
    "$match":{
        "_id":{
            "$oid":"5955f9618c9cdc0010e618f0"
        }
    }
}

The following Java code snippet shows how a query, which uses BSON types, can be converted to an extended JSON query. It therefore utilizes MongoDB’s BSON library. The resulting String extendedOutputQuery could now be used from within the Bosch IoT Insights UI or when calling the REST APIs directly:

String inputQuery = "{$match:{_id:ObjectId(\"5955f9618c9cdc0010e618f0\")}}";
Document bsonDocument = Document.parse( inputQuery );
String extendedOutputQuery = bsonDocument.toJson( new JsonWriterSettings( JsonMode.STRICT ) );
 
// The String extendedOutputQuery results in:
// { "$match" : { "_id" : { "$oid" : "5955f9618c9cdc0010e618f0" } } }