85

I am using a Postgresql database in my rails application. To store large file or data in database I have used blob data type in MySql.

For Postgres which data type I have to use instead of blob in MySql?

2

3 Answers 3

75

use bytea (or Large Objects if you absolutely have to)

6
  • 3
    why are you saying "if you absolutely have to"? Do you see major downsides to using Large Objects?
    – avernet
    Commented Jun 3, 2011 at 2:02
  • 2
    @avernet - yes they are harder to work with and had some security issues
    – user533832
    Commented Jun 3, 2011 at 12:36
  • 18
    @dave no, bytea is binary data - it certainly does not double the size - the 'hex' format is for input and output not for storage. How your client chooses to display that binary data is another matter of course...
    – user533832
    Commented Jul 12, 2012 at 15:06
  • 8
    bytea is loaded when the row is fetched. So be careful about memory. Blob data is loaded when the data is actually needed via STDOUT. Commented Sep 27, 2012 at 5:00
  • 4
    I do find that bytea takes extra RAM on the client-side to convert, but @daveatflow you are wrong about storage. As with all PostgreSQL types, it is exposed to the SQL interface as a text string, but just like timestamps or geometries, or inet addresses, the textual representation is not the internal binary representation. The Pg docs are full of examples of how to write binary times that makes this clear. Commented Apr 10, 2013 at 13:06
23

I think this is the most comprehensive answer on the PostgreSQL wiki itself: https://wiki.postgresql.org/wiki/BinaryFilesInDB

Read the part with the title 'What is the best way to store the files in the Database?'

6

Storing files in your database will lead to a huge database size. You may not like that, for development, testing, backups, etc.

Instead, you'd use FileStream (SQL-Server) or BFILE (Oracle).

There is no default-implementation of BFILE/FileStream in Postgres, but you can add it: https://github.com/darold/external_file

And further information (in french) can be obtained here:
http://blog.dalibo.com/2015/01/26/Extension_BFILE_pour_PostgreSQL.html


To answer the acual question:
Apart from bytea, for really large files, you can use LOBS:

// http://stackoverflow.com/questions/14509747/inserting-large-object-into-postgresql-returns-53200-out-of-memory-error
// https://github.com/npgsql/Npgsql/wiki/User-Manual
public int InsertLargeObject()
{
    int noid;
    byte[] BinaryData = new byte[123];

    // Npgsql.NpgsqlCommand cmd ;
    // long lng = cmd.LastInsertedOID;

    using (Npgsql.NpgsqlConnection connection = new Npgsql.NpgsqlConnection(GetConnectionString()))
    {
        using (Npgsql.NpgsqlTransaction transaction = connection.BeginTransaction())
        {
            try
            {
                NpgsqlTypes.LargeObjectManager manager = new NpgsqlTypes.LargeObjectManager(connection);
                noid = manager.Create(NpgsqlTypes.LargeObjectManager.READWRITE);
                NpgsqlTypes.LargeObject lo = manager.Open(noid, NpgsqlTypes.LargeObjectManager.READWRITE);

                // lo.Write(BinaryData);
                int i = 0;
                do
                {
                    int length = 1000;
                    if (i + length > BinaryData.Length)
                        length = BinaryData.Length - i;

                    byte[] chunk = new byte[length];
                    System.Array.Copy(BinaryData, i, chunk, 0, length);
                    lo.Write(chunk, 0, length);
                    i += length;
                } while (i < BinaryData.Length);

                lo.Close();
                transaction.Commit();
            } // End Try
            catch
            {
                transaction.Rollback();
                throw;
            } // End Catch

            return noid;
        } // End Using transaction 

    } // End using connection

} // End Function InsertLargeObject 



public System.Drawing.Image GetLargeDrawing(int idOfOID)
{
    System.Drawing.Image img;

    using (Npgsql.NpgsqlConnection connection = new Npgsql.NpgsqlConnection(GetConnectionString()))
    {
        lock (connection)
        {
            if (connection.State != System.Data.ConnectionState.Open)
                connection.Open();

            using (Npgsql.NpgsqlTransaction trans = connection.BeginTransaction())
            {
                NpgsqlTypes.LargeObjectManager lbm = new NpgsqlTypes.LargeObjectManager(connection);
                NpgsqlTypes.LargeObject lo = lbm.Open(takeOID(idOfOID), NpgsqlTypes.LargeObjectManager.READWRITE); //take picture oid from metod takeOID
                byte[] buffer = new byte[32768];

                using (System.IO.MemoryStream ms = new System.IO.MemoryStream())
                {
                    int read;
                    while ((read = lo.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        ms.Write(buffer, 0, read);
                    } // Whend

                    img = System.Drawing.Image.FromStream(ms);
                } // End Using ms

                lo.Close();
                trans.Commit();

                if (connection.State != System.Data.ConnectionState.Closed)
                    connection.Close();
            } // End Using trans

        } // End lock connection

    } // End Using connection

    return img;
} // End Function GetLargeDrawing



public void DeleteLargeObject(int noid)
{
    using (Npgsql.NpgsqlConnection connection = new Npgsql.NpgsqlConnection(GetConnectionString()))
    {
        if (connection.State != System.Data.ConnectionState.Open)
            connection.Open();

        using (Npgsql.NpgsqlTransaction trans = connection.BeginTransaction())
        {
            NpgsqlTypes.LargeObjectManager lbm = new NpgsqlTypes.LargeObjectManager(connection);
            lbm.Delete(noid);

            trans.Commit();

            if (connection.State != System.Data.ConnectionState.Closed)
                connection.Close();
        } // End Using trans 

    } // End Using connection

} // End Sub DeleteLargeObject 
3
  • But this is not an answer to the question (it is related, but it is not an answer). Commented Dec 6, 2017 at 9:24
  • 3
    @Bruno Ranschaert: True - fixed that. But neither does the answer with most upvotes. But the acual question is not "how to use blob/bytea/lobs", but "how to store large file or data in database" (as written in the question) - the answer is: large files don't belong into the database - and I mean neither as bytea nor as blob. And my answer answers how to do that (properly). Blob isn't required. Commented Dec 6, 2017 at 10:15
  • Most databases offer blob storage of some kind, so obviously it's a desired feature that many people use.
    – O'Rooney
    Commented Apr 3 at 22:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.