Storing data on Amazon DynamoDB (.NET)

Follow @augustomaia

That's a common question: my app needs to store data for my Viewer models, how can it be done? And this brings a subsequent question: how to ensure proper access permissions?

Based on that we need: index the database with the same identifier used on Forge: the URN; and check if the caller has access to it via the same access token. These are the basic requirements. We may think of additional requirements, such as: we're not using it that often, so why keep it running all the time? Ok, that's a plus, actually, but justifies our architecture.

For this sample let's use Amazon DynamoDB to store our data. Why? Is a no-SQL database, which fits our need of an index-based system that can store any type of data (JSON compatible). That's a huge plus, actually, as a SQL database will be limited to the data predefined type or would need a huge field for random text. And if your app falls into the free tier, that's even better, which is quite big: 200 million requests. See more here. Last, but not least, DynamoDB is managed, so we don't have to worry about scaling it.

Now the computing can also be on-demand with AWS Lambda. Server-less computing is booming right now, and it's easy to understand why: auto-scaling on-demand compute power, and cheap. The free tier includes 1 million calls with 400GB-seconds. 

Let's see the architecture: our JavaScript client app is running Viewer. At some point will need to read (GET) or write (POST/PUT) data. It calls our Lambda function passing the access token and URN, now the Lambda check with Forge if the access token has access to the URN, if so, read/write the data from/to DynamoDB. As any Lambda function, it remains "hot" for a few minutes, and after a while it deallocates. No charge if is not under use.

If you already know AWS Lambda and Amazon DynamoDB there is nothing new here, except how to check for Forge permissions, right? Here is the trick: there is no such an endpoint! But we can simply call a random secured endpoint for it, right? The following .NET code demonstrate the idea. Note it inherits from Controller as a way to be used across all our WebAPI controllers. And, at the comments, you'll notice I did some testing with other endpoints and the metadata seems the fastest here.

public class Security : Controller
{
  private static readonly HttpClient client = new HttpClient();
  private const string KEY = "Authorization";

  protected async Task<bool> IsAuthorized(string urn)
  {
    // Authorization is required
    if (!base.Request.Headers.ContainsKey(KEY))
    {
      base.Response.StatusCode = (int)HttpStatusCode.Unauthorized;
      return false;
    }

    // use Autodesk Forge to get permissions...
    client.DefaultRequestHeaders.Remove(KEY);
    client.DefaultRequestHeaders.Add(KEY, base.Request.Headers[KEY][0]);

    // now we need to call one Forge endpoint to check our credentials
    // the metadata endpoints seems the fastest!
    // https://developer.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-metadata-GET/
    //
    // for the record, also checked manifest, but that's is taking longer to return
    // https://developer.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-manifest-GET/

    // Requesting only the Header seems a few milisecs faster, so let's use it
    HttpRequestMessage req = new HttpRequestMessage();
    req.RequestUri = new System.Uri(
      string.Format("https://developer.api.autodesk.com/modelderivative/v2/designdata/{0}/metadata", urn));
    req.Method = new HttpMethod("GET");
    HttpResponseMessage res = await client.SendAsync(req, HttpCompletionOption.ResponseHeadersRead);

    // a few milisecs slower...
    // HttpResponseMessage response = await client.GetAsync(
    //   string.Format("https://developer.api.autodesk.com/modelderivative/v2/designdata/{0}/metadata", urn));

    if (res.StatusCode != HttpStatusCode.OK)
    {
      base.Response.StatusCode = (int)res.StatusCode;
      return false;
    }

    // good to go!
    return true;
  }
}

Finally, we need a POST & PUT endpoints to save data, and a GET to return data. It needs to be generic and accept any JSON input. The following demonstrates a typical input and output format.

{
    "urn":"dXJuOmFkc2WXlUUXNPd1E",
    "data": {"key": "any valid json"}
} 

And what's the performance? The longest task, in this case, is to call the Forge endpoint to check permissions. In general, on my testing, it takes an average of 500ms to execute in 256Mb memory, but may take up to a few seconds. That should fall into the free tier for 1 million calls. As the routine makes 1 or 2 DynamoDB calls, it should easily fall into the 200 million calls up to 25 read/write capacity units. But don't take my word for it, please review the AWS services pricing! :-)

Ready for the full sample? Check it at this Github repo!