A gigabyte of CSV is roughly five to ten million rows of ordinary tabular data. That number matters more than the gigabyte does, because it is already five times what a spreadsheet can hold in a single sheet, so the first thing to accept is that "open it in Excel" was never on the list.
What is on the list depends on one question: do you need to look at it, or do you need an answer out of it?
- An answer — a total, a filter, a join, a count of duplicates: DuckDB. One command, ten seconds, no import.
- To look at it — scroll, search, eyeball the shape of the data: a viewer
built to stream. EmEditor on Windows,
lessanywhere, a phone app if that's where the file is. - To edit and save it back: hardest case, fewest good answers. See below.
The comparison
Times are for a 1GB, ~8 million row file on an ordinary 16GB laptop with an SSD. They are indicative, not benchmarks. Your columns and your disk will move them around. The column that doesn't move is the last one.
| Tool | Opens 1GB? | Time to first row | Peak RAM | Cost |
|---|---|---|---|---|
| Excel | No — 1,048,576-row ceiling | — | — | Paid |
| Excel + Power Query | Yes, as a load step | Minutes | High | Paid |
| Google Sheets | No — 10M cells, ~100MB upload | — | — | Free |
| Numbers | No — 1M rows, and slower first | — | — | Free |
| DuckDB | Yes | ~1s | ~200MB | Free |
| Python + pandas | Yes, with chunksize | 30s+ | 5–10GB if naive | Free |
| Python + Polars | Yes | ~5s | ~1.5GB | Free |
| EmEditor (Windows) | Yes, comfortably | ~5s | Low | Paid |
| VS Code | Struggles | 30s+ | High | Free |
less / head / awk | Yes | Instant | ~0 | Free |
| OpenRefine | Yes, slowly | Minutes | High | Free |
| Phone CSV app | Yes, if built for it | Seconds | Managed | Free tier |
Two rows there are traps. Google Sheets advertises 10 million cells, which sounds close, but a 20-column file hits that at 500,000 rows, and the upload limit stops you long before either. pandas will happily open the file and then use eight times its size in RAM, because it stores every text column as Python objects. That's the swap-thrashing "it opened but the laptop is dead" experience people report.
By what you're actually doing
You need an answer
DuckDB, and it isn't close:
duckdb -c "SELECT region, COUNT(*), SUM(amount) FROM 'big.csv' GROUP BY region"
It reads the file in parallel, streams rather than loading, and never allocates much more than the working set of your query. A GROUP BY over 8 million rows finishes in a couple of seconds. The full treatment (type overrides, joining two files, writing results back out) is in running SQL queries on a CSV without a database.
If you'd rather stay in Python, use Polars with scan_csv (lazy: it plans
the whole query before reading) or pandas with an explicit chunksize and a
dtype map. Naive pd.read_csv('big.csv') is the thing to avoid.
You need to look at it
On Windows, EmEditor. It is the one large-file editor that holds up, and it opens multi-GB CSVs as a grid with sort and filter. It's paid; it earns it if this is a recurring problem.
Anywhere, the shell. Underrated for the first five minutes with any large file:
head -5 big.csv # what are the columns
wc -l big.csv # how many rows really
awk -F, 'NR<=1000' big.csv > sample.csv # a slice you can open normally
less -S big.csv # scroll it, no line wrapping
Making a 1,000-row sample and opening that in Excel answers "what's in this file" faster than any tool that opens the whole thing.
Not VS Code, and not Notepad++. Both load the entire file into memory as text, and both will do it, just slowly, with a warning, and no useful table view at the end.
The file is on your phone
The 1GB case is hard here: phones have less RAM than laptops and stricter limits on what an app may hold. Nothing on a phone opens a full gigabyte comfortably, so test any app claiming otherwise before you trust it. What phone apps do well is the range just below: the 100–500MB, several-hundred-thousand-row exports that a phone spreadsheet silently truncates.
Opens large files without truncating them, then answers questions about the data with SQL or in plain language, which is usually what you wanted from a file too big to scroll. iOS and Android.
What it doesBefore trusting any phone app with a file that matters, run the check in the last section of this page.
You need to edit it and save it back
This is the case with the fewest honest answers, because "edit" means holding the file and writing it out again.
- A few surgical changes — fix a value, drop a column: do it with DuckDB or
awkand write a new file. It is faster than any interactive tool and it leaves the original intact. - Real interactive editing on Windows: EmEditor.
- Real interactive editing elsewhere: split the file first.
split -l 1000000 big.csv part_gives you pieces a spreadsheet can hold, at the cost of having to reassemble them (catworks, but delete the repeated header rows). - Under about 500,000 rows, a phone editor handles it directly:
Spreadsheet-style editing on 500,000-row files with every row written back on save, the free version included. iOS and iPadOS.
What it doesConverting the problem away
If you'll touch the file more than twice, stop treating it as a CSV:
duckdb -c "COPY (SELECT * FROM 'big.csv') TO 'big.parquet'"
Parquet is columnar, typed and compressed. The 1GB CSV becomes 100–200MB, queries that scan a few columns get five to twenty times faster, and the types you fought over are now stored in the file. DuckDB, Polars and pandas all read it natively. This is the single highest-value ten seconds in this article.
Testing whether a tool actually opened it
Plenty of tools open a large CSV by opening part of it and not mentioning which part. Before you draw a conclusion from a file, check:
- Count the rows first, outside the tool:
wc -l big.csv. Subtract one for the header. - Go to the last row in the tool and compare the count.
- Check the last row's contents against
tail -1 big.csv.
If those disagree, the tool truncated. Any conclusion you draw (a total, a max, "there are no orders from that region") is now wrong in a way nothing on screen tells you. Silent truncation is the actual danger with large files, more than crashes are: a crash is at least honest.
If it's specifically Excel that failed
Two neighbouring problems that are easy to confuse. If Excel refused the file or showed only part of it, that's the row ceiling, and how to open a large CSV file that Excel won't open covers the routes around it. If Excel opened the file and then froze, hung or died, that's a different fault with different causes, covered in why Excel crashes on large files.