Uploading files to MongoDB GridFS

Almost at the same time, I silently celebrate the first birthday of my blog. My first article appeared a little over a year ago. It is great to see how Symfony2 has become more and more popular during these twelve months. Your comments and visits encourage me to keep posting articles. So, thank you all! And thanks, Dennis, for contributing.

GridFS is a specification for storing large files in MongoDB. In this post I will explain how you can easily upload a file to GridFS and then retrieve it from the database to serve it to the browser.

But before we get started a short explanation of GridFS and this is how Kristina Chodorow of 10Gen explains it:

GridFS breaks large files into manageable chunks. It saves the chunks to one collection (fs.chunks) and then metadata about the file to another collection (fs.files). When you query for the file, GridFS queries the chunks collection and returns the file one piece at a time.

Of course the MongoDB PHP Driver comes with a couple of classes that can be used for storing and retrieving files from GridFS.

Some other advantages of GridFS are pointed out in this post:

• If you are using replication or autosharding your GridFS files will be seamlessly sharded or replicated for you.

• MongoDB datafiles are broken into 2 GB chunks so MongoDB will automatically break your files into OS manageable pieces.

• You won’t have to worry about OS limitations like ‘weird’ filenames or a large number of files in one directory, etc.

• MongoDB will auto generate the MD5 hash of your file and store it in the file’s document. This is useful to compare the stored file with it’s MD5 hash to see if it was uploaded correctly, or already exists in your database.

Defining a GridFS Document

We will start with a simple Upload Document:

namespace Dennis\UploadBundle\Document;

use Doctrine\ODM\MongoDB\Mapping\Annotations as MongoDB;

/**
 * @MongoDB\Document
 */
class Upload
{
    /** @MongoDB\Id */
    private $id;

    /** @MongoDB\File */
    private $file;

    /** @MongoDB\String */
    private $filename;

    /** @MongoDB\String */
    private $mimeType;

    /** @MongoDB\Date */
    private $uploadDate;

    /** @MongoDB\Int */
    private $length;

    /** @MongoDB\Int */
    private $chunkSize;

    /** @MongoDB\String */
    private $md5;

    public function getFile()
    {
        return $this->file;
    }

    public function setFile($file)
    {
        $this->file = $file;
    }

    public function getFilename()
    {
        return $this->filename;
    }

    public function setFilename($filename)
    {
        $this->filename = $filename;
    }

    public function getMimeType()
    {
        return $this->mimeType;
    }

    public function setMimeType($mimeType)
    {
        $this->mimeType = $mimeType;
    }

    public function getChunkSize()
    {
        return $this->chunkSize;
    }

    public function getLength()
    {
        return $this->length;
    }

    public function getMd5()
    {
        return $this->md5;
    }

    public function getUploadDate()
    {
        return $this->uploadDate;
    }
}

The imported bit in this definition is the @MongoDB\File annotation. It tells Doctrine MongoDB ODM that the document needs to be stored using GridFS and that the MongoGridFSFile instance is placed into the $file property.

The properties $chunkSize, $length, $md5 and $uploadDate don’t need a setter because they will be filled automatically by the MongoDB Driver.

Processing an uploaded file

As an example I will use a simple controller which uses the form builder to create a single form containing a ‘file’ Field Type:

namespace Dennis\UploadBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;

class UploadController extends Controller
{
    public function newAction(Request $request)
    {
        $form = $this->createFormBuilder(array())
            ->add('upload', 'file')
            ->getForm();

        if ($request->isMethod('POST')) {
        	$form->bind($request);

            // ...
        }

        return array('form' => $form->createView());
    }
}

My goal is to save the file to the database directly from the /tmp directory, where the file is placed by default after uploading, to prevent multiple movements across the filesystem before saving it to the database. To achieve this I will retrieve the submitted form data via $form->getData() to obtain the UploadedFile object. When using an Entity you could obtain the UploadedFile object by getting the property value that has been defined as a ‘file’ on the form builder.

The UploadedFile object contains all the information we need to add the file to the database directly from its temporary location because it is based upon the information of the PHP $_FILES global.

use Dennis\UploadBundle\Document\Upload;

public function newAction(Request $request)
{
	// ...

	$data = $form->getData();

	/** @var $upload \Symfony\Component\HttpFoundation\File\UploadedFile */
	$upload = $data['upload'];

	$document = new Upload();
	$document->setFile($upload->getPathname());
	$document->setFilename($upload->getClientOriginalName());
	$document->setMimeType($upload->getClientMimeType());

	$dm = $this->get('doctrine.odm.mongodb.document_manager');
	$dm->persist($document);
	$dm->flush();
}

Now that we have all the necessary information we can create an Upload object where we will set the $file property to the pathname of the temporary file location. The UploadedFile object also provides us with some additional information about the file and we will add some of this information to our Upload Document, such as filename and MIME type. At this moment the Document is ready to be saved to the database and as you would expect from the ODM, this is done by a regular persist() and flush().

Retrieving the uploaded file from GridFS

So, now that the file is savely stored in the database we will take a look on how to get the file out of it and serve it to the browser.

I have added an extra action method, to my previous defined controller:

/**
 * @Route("/{id}", name="upload_show")
 */
public function showAction($id)
{
    $upload = $this->get('doctrine.odm.mongodb.document_manager')
        ->getRepository('DennisUploadBundle:Upload')
        ->find($id);

    if (null === $upload) {
        throw $this->createNotFoundException(sprintf('Upload with id "%s" could not be found', $id));
    }

    $response = new Response();
    $response->headers->set('Content-Type', $upload->getMimeType());

    $response->setContent($upload->getFile()->getBytes());

    return $response;
}

Pretty straightforward steps as you may see. The id generated by MongoDB should be provided in the url and will be used for retrieving the Upload Document from the repository. A new Response object will be created where the Content-Type is set to the MIME type of the Upload Document. The content of the file can be retrieved from the $file property by calling getBytes() and can be added to the Response with setContent().

Stream resource & StreamedResponse

As of version 1.3.0-beta1 of the MongoDB Driver there will be a getResource() method available which returns a stream resource of the file. This enables you to use the StreamedResponse object instead of the regular Response object. The StreamedResponse object allows you to stream content back to the client by defining a callback and will look something like this:

public function showAction($id)
{
    // ...

    $response = new StreamedResponse();
    $response->headers->set('Content-Type', $upload->getMimeType());

    $stream = $upload->getFile()->getResource();

    $response->setCallback(function () use ($stream) {
        fpassthru($stream);
    });

    return $response;
}

This will be it for now! In my next post I will write about how you can combine a Upload Document by creating a custom Entity Type.

• Controller
• GridFS
• MongoDB