September 12, 2022

Advanced Query with Model Derivative API

Our Derivative Services team continues to improve the capabilities of the Model Derivative service. We recently introduced the ability to perform more granular queries on the properties of a model under a public beta release. You can find the docs here, and a deployed sample here, with source code here. Please provide comments and feedback here. For general help requests, please use the Forge Help.

To get started, let’s look at the Model Derivative (MD) service in Forge, and how it enables users to represent and share the designs in a format of structured metadata. One of the metadata structures is object properties. MD API provides the way to fetch properties on the cloud, without the Forge Viewer . The endpoint GET: {urn}/metadata/{guid}/properties returns the list of properties for each object contained in the model view. By default, it will return properties of all objects in JSON in one chunk. Obviously, the larger size of the model, the more data that will be extracted.

Therefore, the query parameter forceget was introduced a while back. It forcibly fetches the data when it exceeded 20M. Even with this query parameter, the performance may suffer because of the massive amount of data to be transferred  when it is a large model. Also there may be more data than what you care about. So, a workaround was to download the sql-lite database (also extracted by MD service), but this means additional step, and disconnection from the current data on the cloud. In addition, in many cases, the users only want to filter objects that contain specific properties, or find limit objects that have relationships, or check objects that match a formula and so on.

The newly released endpoint supports advanced query options and returns paginated results!

             POST {urn}/metadata/{modelGuid}/properties:query

The new endpoint allows you to specify the request body with the items below:

  • Pagination: this new endpoint supports to return JSONdata in pagination. The limit specifies max 20-1000 objects in one page, and offset specifies the number of properties in this page
  • Query: a customized query DSL(Domain Specific Languages) defined in JSON format. It defines the clause with the conditions in an sql-like format. Several conditions are supported.
  • Fields: keyword defines a set of object fields that will be included in the response body.
  • Payload: specifies the format for numeric values in the response body. The format can be in text (the default), or in the format of ##<VALUE_OF_PROPERTY><UNIT_OF_VALUE><PRECISION><SYSTEM_UNIT>

In this blog, we will introduce Query. To begin with, let us take a look at a sample properties of the object:

1

Inside the properties, we call the “Dimensions” as a direct category, the “Area” as a leaf property. The “Version”, “Component Name” as a direct child property. Please note, the value type of the properties is basically a JSON that contains multiple nested key-value pairs. To locate the specific property, a JSON-path styled key is composed like properties.{category}.{property}. e.g. we can use properties.Dimensions.Area to refer to “Area” property. Query supports a search by basic attributes, direct child property and leaf property.

In Query, you can define the clause that is to match the condition. This new endpoint accepts a few conditions(rules) such as $contains, $gt, $between etc.  Note: In current release, only one clause (with one condition) is supported in the request body.

  • $in: this is a clause of a list query. It contains a list of exact values for a specific field. The query results returns objects that must contain the specified field and the values of the filed should match one of the defined values. The basic attribute objectId (numeric) and externalId (string) are supported in this release. For example, the below query will return those objects whose values are 123 or 789 :
    { 
         "$in": ["objectid",123, 789 ] 
    }
  • $prefix: this is a clause to match the objects whose field contains a specific prefix string (case insensitive). Only the basic attribute name is supported. e.g. if to search the object whose name starts with “System Panel”, define the clause like:
    {
       "$prefix": ["name", 'System Panel']
    }
  • $contains: this clause specifies multiple terms for a specific property. The result returns objects that should contain the specified field and values of the field should contain one or more terms defined(case insensitive). Each term needs be separated by the ASCII whitespace. The maximum of 50 terms is allowed in one query. e.g. in the demo below, the text matching rule “Aluminum Steel” will be converted into separate terms: Aluminum and Steel. So the above query means to return objects whose properties JSON should contain a leaf property (Material ) and the value of the value should contain sub string “Aluminum” or “Steel”, or both.
     {
        "$contains":["properties.Constrains.Material", "Aluminum Steel"]
     }
  • $eq: this condition accepts the basic attributes name, or a unit property. If it is name, the condition is to match the string of the name exactly. e.g.
       {  
         "$eq":["name ", “Door [34567]” ]
       }

           If it is a unit property, the second value must be numeric. In addition, the value is always in standard (united) unit.e.g. length unit is meter (m). Assume the exact value in source                   model is in centimeter (cm), the value is 500. To use $eq to find the object whose properties.Constrains.height =500, you must use standard unit (value =5) to search.

      {  
          "$eq":["properties.Constrains.height", 5]
      }
  •  $between: this condition accepts a unit property only. It returns the objects whose value is between two values( boundary values are included). The same to $eq, it searches by standard(united) unit. e.g. find objects with height that  is >=34.567 meter  and <=123.789 meter:
    {
       "$between": ["properties.Constrains.height ",34.567, 123.789]
    }
  •  $le query:  this condition accepts a unit property only. It returns the objects whose value is less than a value ( boundary values are included). The same, it searches by standard(united) unit. e.g. find objects with height that <=123.789 meter:
      {
           "$le": ["properties.Constrains.height",123.789]
      }
  •  $ge query: this condition accepts a unit property only. It returns the objects whose value is less than a value ( boundary values are included). The same, it searches by standard(united) unit. e.g. find objects with height that  >=34.567  meter:
       {
          "$ge": ["properties.Constrains.Length",34.567]
       }

In the internal mechanism, all field of any types will have string index for $contains. If the original type is numeric, it will be also indexed for $le,$ge, $between, $eq. 

We are also working on the demo samples how to play with this API. Stay tuned. In addition, in the next blog, we will introduce Fields and Payload

Posted by

Xiaodong Liang has been with Autodesk since 2007, focusing on providing programming support, consulting, training and evangelism to external developers. He started his career China and now lives in Beijing, China. Xiaodong is currently a developer consultant in the team Developer Technical Services (DevTech), the worldwide team of API gurus providing technical services through the Autodesk Developer Network. He supports...

Kevin Vandecar

Principal Developer Advocate

Kevin Vandecar is a Forge developer advocate and also the manager for the Media & Entertainment and Manufacturing Autodesk Developer Network Workgroups. His current specialty is 3ds Max software customization and programming areas. He is also working on  web development skills and exploring areas in Three.js. He is based in Manchester, NH, US. 

Related Posts