SchwertImStein@lemmy.dbzer0.com to Programmer Humor@programming.devEnglish · 2 months agomuskrat's data eng expert's hard drive overheats while processing 60k rowslemmy.dbzer0.comimagemessage-square296fedilinkarrow-up1988arrow-down114
arrow-up1974arrow-down1imagemuskrat's data eng expert's hard drive overheats while processing 60k rowslemmy.dbzer0.comSchwertImStein@lemmy.dbzer0.com to Programmer Humor@programming.devEnglish · 2 months agomessage-square296fedilink
minus-squarenaught@sh.itjust.workslinkfedilinkarrow-up59·2 months agoi mean its even excel sized depending on how many columns. This is seriously sad and alarming
minus-squarewise_pancake@lemmy.calinkfedilinkarrow-up3·2 months agoHoly shit if this ids lm issue that’s too funny
minus-squaremorrowind@lemmy.mllinkfedilinkarrow-up50·2 months agoSqlite can easily handle millions of rows. Don’t sell it short
minus-squareThe_Decryptor@aussie.zonelinkfedilinkEnglisharrow-up9·2 months agoHow about a 6.4TB sqlite database? https://boyter.org/posts/searchcode-bigger-sqlite-than-you/
minus-squareKorhaka@sopuli.xyzlinkfedilinkEnglisharrow-up4·2 months agoI have an sqlite db that is a few GB in size, game saves using the format. Sadly almost all blob data, would love to play with it if it was a bit more readable
deleted by creator
i mean its even excel sized depending on how many columns. This is seriously sad and alarming
deleted by creator
Holy shit if this ids lm issue that’s too funny
Sqlite can easily handle millions of rows. Don’t sell it short
How about a 6.4TB sqlite database?
https://boyter.org/posts/searchcode-bigger-sqlite-than-you/
Should be enough to hold 60k rows
I have an sqlite db that is a few GB in size, game saves using the format. Sadly almost all blob data, would love to play with it if it was a bit more readable
deleted by creator
60k is single json file