15 Mar 2014

Handling Blobs in MongoDB


When we talk of database one question that arises is how do we store large objects in MongoDB, and when i say large objects i am necessarily referring to any data that is larger than 16MB, can also be referred most commonly as Blobs  (remember a document size in MongoDB cannot be larger than 16MB). The answer is GridFS (Grid File System).

Now as we know that any document cannot be larger than 16MB of size in MongoDB, what GridFS does is that it breaks a large file into chunks and stores them into two different collections. One collection is called the chunks collection and the other as file collection.
How MONGODB saved BLOB using GridFS

Lets now see how we actually do save a simple large file in mongodb using java and how does it gets saved in the mongodb instance.

I am going to create a very simple java file which will save a large zip file using gridfs

package course;

import java.io.FileInputStream;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.List;

import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.MongoClient;
import com.mongodb.MongoClientURI;
import com.mongodb.gridfs.GridFS;
import com.mongodb.gridfs.GridFSInputFile;

public class GridFSExample {
public static void main(String[] args) throws UnknownHostException {
	MongoClient client = new MongoClient(new MongoClientURI("mongodb://localhost"));
	DB db = client.getDB("mycollection");
	FileInputStream fis = null;
	GridFS docs = new GridFS(db, "docs");
		fis = new FileInputStream("D:/_handouts/final_exam.zip");
	}catch (Exception e) {
	GridFSInputFile doc = docs.createFile(fis, "final.zip");
	BasicDBObject meta = new BasicDBObject("description","Test document");
	List tags = new ArrayList();
	meta.append("tags", tags);
Lets me try to explain the code a little, although its quite self explanatory. In this example the only difference that you would note while storing a large file is using GridFS and GridFSInputFile API's. Also you can set some meta tag that you want to define for the document.

Once i have done it let me try and show you how did it get stored in the MongoDB. Let us see what all gets created in my DB mycollection
Figure 2: Console screenshot showing collections in DB mycollections
Now if you note that there is no collection as docs that got created, instead there are docs.files and docs.chunks that was created. Lets get deeper, what are the contents of the docs.chunks and docs.files contents

Figure 3: Screenshot showing the contents of docs.files collection
Lets now see whats in the docs.chunks collection
Figure 4: Screenshot showing contents of docs.chunks collection

Now lets drive the analogy of the contents of the docs.files and docs.chunks collection, if you see the metadata and the basic details of the file got into the docs.files collection, however the real content of the blob, actually got stored in the docs.chunks collections as multiple documents with the same value of files_id and a counter "n", which tells how the chunks should be assembled while fetching the file back.

Hope this would make your life simpler now dealing with large data in mongodb.

Written by

We are Creative Blogger Theme Wavers which provides user friendly, effective and easy to use themes. Each support has free and providing HD support screen casting.


Post a Comment


© 2013 NimbleGeek. All rights resevered. Designed by Templateism

Back To Top