Database Dilemma... Please help.

[Blake B.]

On Apr 6, 2006, at 11:23 AM, Nick Barker wrote:

> The question is how wide are your rows and how well do you index.
> Personally I find this the biggest issue on speed with any database.
> Good luck finding someone that has 100 million records.  I cannot help
> you there.

Craigslist has 100,000,000 rows in their archive database.  They list  
it in this case study:  http://www.mysql.com/it-resources/case- 
studies/mysql-craigslist-casestudy.pdf

-Blake

>
> On 4/6/06, Jason Jones <poeticintensity at gmail.com> wrote:
>> I recently got hired as the resident Linux-Geek for a new company  
>> headed by
>> a guy who has created 5 of the fortune 500 companies.  My co- 
>> worker (we'll
>> call Fred) recently got hired as well....  Fred has 9 years of MS- 
>> SQL DBA
>> experience.
>>
>> We have a situation where we're using MySQL 5.0 and are only  
>> dealing with
>> very limited, read "around 100Megs" amounts of data which will  
>> surely grow
>> to more than 100 million rows of data shortly.
>>
>>   Fred is luckily open-minded enough to accept the fact that MySQL  
>> ($0.00)
>> is better than MS-SQL ($15,000.00) at the current time due to our  
>> lack of
>> data.  However....  He's pretty convinced that this is surely not  
>> going to
>> be the case when the data grows.
>>
>> Fred has concrete evidence of his ability to handle more than 100  
>> million
>> rows of data per table with MS-SQL with little to no loss of speed.
>>
>> I'm dead set on keeping my OSS databases, but am having a hard  
>> time finding
>> concrete evidence that either Postgres OR MySQL can handle more  
>> than 100
>> million rows of data per table without suffering speed hits.
>>
>> Can anyone here point me to something, somewhere that gives  
>> numbers on any
>> OSS datbase handling that amount of data and maintaining good  
>> numbers on
>> speed, with possible hints as to its configuration?
>>
>> I've personally never handled any OSS db with more than a couple  
>> hundred
>> thousand rows TOTAL, (but have around 3 years exp. handling many  
>> various
>> smaller dbs) and am kind of twitchy about what's going to happen  
>> with our db
>> as it grows exponentially to hundreds of millions of rows.
>>
>> Hardware is not an issue.  Disk space is not an issue.  The only  
>> issue is
>> whether MySQL (or PostgreSQL) can be properly configured to handle  
>> hundreds
>> of millions of rows per table without hacking it into some  
>> slashdot-esque
>> frankenstein configuration.
>>
>> Any takers for this one?  I'm kind of scared I'm going to lose the  
>> CEO on
>> this battle and switch to MS-SQL.... I'm dealing with a guy who is  
>> extremely
>> competent in MS-SQL and has demonstrated abilities to handle any  
>> amount of
>> data.  If I can demonstrate the same ability with an OSS solution,  
>> I'm sure
>> I'll win and keep the OSS solution, due to the obvious financial  
>> advantages.
>>
>> Thanks anyone who points me to any helpful information.
>>
>> --Jason
>>
>> PS - I have a pretty good amount of experience with MySQL, but am  
>> certain
>> PostgreSQL is just as good.  If information can be given about  
>> *any* OSS db
>> solution, I'd be most grateful.  Thank you.
>>
>> /*
>> PLUG: http://plug.org, #utah on irc.freenode.net
>> Unsubscribe: http://plug.org/mailman/options/plug
>> Don't fear the penguin.
>> */
>>
>
>
> --
> Nick Barker
> 785-3824
>
> /*
> PLUG: http://plug.org, #utah on irc.freenode.net
> Unsubscribe: http://plug.org/mailman/options/plug
> Don't fear the penguin.
> */