9 Mar 2017

C# Resumable Upload File by Forge SDK

Default blog image

Any file could be uploaded to Forge by direct uploading of Data Manangment API :

PUT buckets/:bucketKey/objects/:objectName

While when it is a large file, it is not a safe way to upload directly. If the internet connection is cranky, the uploading would be interrupted, which causes all previous jobs will be in vain. So it is strongly recommended to use resumable uploading

PUT buckets/:bucketKey/objects/:objectName/resumable

The basic logic with the method is to devide the large file to pieces (chunks), then upload the piece one by one. The paramter of this HTTP request (Session-Id) is a string  that uniquly identifies a session of a file being uploaded, by which Forge can know combine the pieces together. 

Forge SDK  provides the convenient method for resumble uploading. e.g. C# SDK. The test sample  has only the demo on direct uploading. So I added some codes based on the sample to do resumable uploading. 

 

private static dynamic resumableUploadFile()
    {
        Console.WriteLine("*****begin uploading large file");
        string path = FILE_PATH;
        if (!File.Exists(path))
            path = @"..\..\..\" + FILE_PATH;

        //total size of file        
        long fileSize = new System.IO.FileInfo(path).Length;
        //size of piece, say 2M    
        long chunkSize = 2 * 1024 * 1024 ;
        //pieces count
        long nbChunks = (long)Math.Round(0.5 + (double)fileSize / (double)chunkSize);
        //record a global response for next function. 
        ApiResponse<dynamic> finalRes = null ;
        using (FileStream streamReader = new FileStream(path, FileMode.Open))
        {
            //unique id of this session
            string sessionId = RandomString(12);
            for (int i = 0; i < nbChunks; i++)
            {
                //start binary position of one certain piece 
                long start = i * chunkSize;
                //end binary position of one certain piece 
                //if the size of last piece is bigger than  total size of the file, end binary 
                // position will be the end binary position of the file 
                long end = Math.Min(fileSize, (i + 1) * chunkSize) - 1;

                //tell Forge about the info of this piece
                string range = "bytes " + start + "-" + end + "/" + fileSize;
                // length of this piece
                long length = end - start + 1; 

                //read the file stream of this piece
                byte[] buffer = new byte[length];
                MemoryStream memoryStream = new MemoryStream(buffer);

                int nb = streamReader.Read(buffer, 0, (int)length);
                memoryStream.Write(buffer, 0, nb);
                memoryStream.Position = 0;

                //upload the piece to Forge bucket
                ApiResponse<dynamic> response = objectsApi.UploadChunkWithHttpInfo(BUCKET_KEY,
                        FILE_NAME, (int)length, range, sessionId, memoryStream,
                        "application/octet-stream"); 

                finalRes = response;

                if (response.StatusCode == 202){
                    Console.WriteLine("one certain piece has been uploaded");
                    continue;
                }
                else if(response.StatusCode == 200){
                    Console.WriteLine("the last piece has been uploaded");
                }
                else{
                    //any error
                    Console.WriteLine(response.StatusCode);
                    break;

                } 
            } 
                   
        }
                        
        return (finalRes);
    }

 

Related Article