0x0023 - Mysql - Throttling at 500 SQL Statements Per Second.

So we have a giant 1 minute quote base of approximately 1.6 Billion quotes to process.

• We write a python loading class and it runs at about 500 quotes every 5-6 seconds.
• Great a basic calculation shows we will take 6 months to reload this giant database of 5000 stocks.
• We then add filtering so that paraellel threads can all share a piece of the work.
• At 14 parellel threads the speed is unchanged!

In the above code snippet we have launched 14 parallel threads all working on 1 letter of the alphabet - with almost no change in speed.