Databasing over 12Gig of images

Questions about the MySQL, PostgreSQL, and most other databases, as well as using it with PHP can be asked here.

Moderator: General Moderators

Post Reply
xrob
Forum Newbie
Posts: 5
Joined: Tue Jan 18, 2005 12:21 pm

Databasing over 12Gig of images

Post by xrob »

Hey

I have a lot of images, pdfs, and smallish (50meg max) videos, i dont suppose anyone knows if mysql would be stable enough hold all this data. if not is there any database strong enough to withstand 1,500+ hits a day.

cheers
rob
User avatar
JayBird
Admin
Posts: 4524
Joined: Wed Aug 13, 2003 7:02 am
Location: York, UK
Contact:

Post by JayBird »

Why would you want to store it ALL in the database?

Why not store the content in the File System, then just have references in the database i.e. store the path to the content
User avatar
s.dot
Tranquility In Moderation
Posts: 5001
Joined: Sun Feb 06, 2005 7:18 pm
Location: Indiana

Post by s.dot »

Forgive me if I'm wrong (I haven't read up on this topic) but I believe the most efficient way to do what you're wanting is by doing what weirdan said.

Also, if you did try to put in all as binary data in the database, databases DO have data size limits that depend on which operating system you're using.

I also think mysql has a table size limit of 4GB (could be wrong, again).
Set Search Time - A google chrome extension. When you search only results from the past year (or set time period) are displayed. Helps tremendously when using new technologies to avoid outdated results.
User avatar
JayBird
Admin
Posts: 4524
Joined: Wed Aug 13, 2003 7:02 am
Location: York, UK
Contact:

Post by JayBird »

scottayy wrote:...by doing what weirdan said.
What who said? :P
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Post by Weirdan »

Pimptastic wrote: What who said? :P
:P
User avatar
s.dot
Tranquility In Moderation
Posts: 5001
Joined: Sun Feb 06, 2005 7:18 pm
Location: Indiana

Post by s.dot »

Aha. I had recently posted a topic and weirdan replied to it. So I must've had his name stuck in my head.
Set Search Time - A google chrome extension. When you search only results from the past year (or set time period) are displayed. Helps tremendously when using new technologies to avoid outdated results.
User avatar
pickle
Briney Mod
Posts: 6445
Joined: Mon Jan 19, 2004 6:11 pm
Location: 53.01N x 112.48W
Contact:

Post by pickle »

I fully agree. I've got a student image database with the imag data stored in the database itself. All told, there are about 1000 images - each < 20K each. I can really notice the delay caused by having to pull out all the image data from the database - I'll join the mob and suggest you store the files in the filesystem. If you're worried about controlling access to these files, just put them outside your web root, and use an intermediate php file to serve them up.
Real programmers don't comment their code. If it was hard to write, it should be hard to understand.
User avatar
s.dot
Tranquility In Moderation
Posts: 5001
Joined: Sun Feb 06, 2005 7:18 pm
Location: Indiana

Post by s.dot »

1.4.4. How Large MySQL Tables Can Be
MySQL 3.22 had a 4GB (4 gigabyte) limit on table size. With the MyISAM storage engine in MySQL 3.23, the maximum table size was increased to 65536 terabytes (2567 – 1 bytes). With this larger allowed table size, the maximum effective table size for MySQL databases is usually determined by operating system constraints on file sizes, not by MySQL internal limits.

The InnoDB storage engine maintains InnoDB tables within a tablespace that can be created from several files. This allows a table to exceed the maximum individual file size. The tablespace can include raw disk partitions, which allows extremely large tables. The maximum tablespace size is 64TB.

The following table lists some examples of operating system file-size limits. This is only a rough guide and is not intended to be definitive. For the most up-to-date information, be sure to check the documentation specific to your operating system.

Operating System File-size Limit
Linux 2.2-Intel 32-bit 2GB (LFS: 4GB)
Linux 2.4+ (using ext3 filesystem) 4TB
Solaris 9/10 16TB
NetWare w/NSS filesystem 8TB
Win32 w/ FAT/FAT32 2GB/4GB
Win32 w/ NTFS 2TB (possibly larger)
MacOS X w/ HFS+ 2TB

On Linux 2.2, you can get MyISAM tables larger than 2GB in size by using the Large File Support (LFS) patch for the ext2 filesystem. On Linux 2.4, patches also exist for ReiserFS to get support for big files (up to 2TB). Most current Linux distributions are based on kernel 2.4 or higher and include all the required LFS patches. With JFS and XFS, petabyte and larger files are possible on Linux. However, the maximum available file size still depends on several factors, one of them being the filesystem used to store MySQL tables.
Taken from http://dev.mysql.com/doc/refman/5.0/en/table-size.html
Set Search Time - A google chrome extension. When you search only results from the past year (or set time period) are displayed. Helps tremendously when using new technologies to avoid outdated results.
Post Reply