mygems

How to open a 1GB CSV file (2026 guide)

A gigabyte is where most tools stop being honest about what they can do. Here is what opens a 1GB CSV, what it costs in RAM, and what quietly truncates.

By uos ·

A gigabyte of CSV is roughly five to ten million rows of ordinary tabular data. That number matters more than the gigabyte does, because it is already five times what a spreadsheet can hold in a single sheet, so the first thing to accept is that "open it in Excel" was never on the list.

What is on the list depends on one question: do you need to look at it, or do you need an answer out of it?

  • An answer — a total, a filter, a join, a count of duplicates: DuckDB. One command, ten seconds, no import.
  • To look at it — scroll, search, eyeball the shape of the data: a viewer built to stream. EmEditor on Windows, less anywhere, a phone app if that's where the file is.
  • To edit and save it back: hardest case, fewest good answers. See below.

The comparison

Times are for a 1GB, ~8 million row file on an ordinary 16GB laptop with an SSD. They are indicative, not benchmarks. Your columns and your disk will move them around. The column that doesn't move is the last one.

ToolOpens 1GB?Time to first rowPeak RAMCost
ExcelNo — 1,048,576-row ceilingPaid
Excel + Power QueryYes, as a load stepMinutesHighPaid
Google SheetsNo — 10M cells, ~100MB uploadFree
NumbersNo — 1M rows, and slower firstFree
DuckDBYes~1s~200MBFree
Python + pandasYes, with chunksize30s+5–10GB if naiveFree
Python + PolarsYes~5s~1.5GBFree
EmEditor (Windows)Yes, comfortably~5sLowPaid
VS CodeStruggles30s+HighFree
less / head / awkYesInstant~0Free
OpenRefineYes, slowlyMinutesHighFree
Phone CSV appYes, if built for itSecondsManagedFree tier

Two rows there are traps. Google Sheets advertises 10 million cells, which sounds close, but a 20-column file hits that at 500,000 rows, and the upload limit stops you long before either. pandas will happily open the file and then use eight times its size in RAM, because it stores every text column as Python objects. That's the swap-thrashing "it opened but the laptop is dead" experience people report.

By what you're actually doing

You need an answer

DuckDB, and it isn't close:

duckdb -c "SELECT region, COUNT(*), SUM(amount) FROM 'big.csv' GROUP BY region"

It reads the file in parallel, streams rather than loading, and never allocates much more than the working set of your query. A GROUP BY over 8 million rows finishes in a couple of seconds. The full treatment (type overrides, joining two files, writing results back out) is in running SQL queries on a CSV without a database.

If you'd rather stay in Python, use Polars with scan_csv (lazy: it plans the whole query before reading) or pandas with an explicit chunksize and a dtype map. Naive pd.read_csv('big.csv') is the thing to avoid.

You need to look at it

On Windows, EmEditor. It is the one large-file editor that holds up, and it opens multi-GB CSVs as a grid with sort and filter. It's paid; it earns it if this is a recurring problem.

Anywhere, the shell. Underrated for the first five minutes with any large file:

head -5 big.csv                 # what are the columns
wc -l big.csv                   # how many rows really
awk -F, 'NR<=1000' big.csv > sample.csv   # a slice you can open normally
less -S big.csv                 # scroll it, no line wrapping

Making a 1,000-row sample and opening that in Excel answers "what's in this file" faster than any tool that opens the whole thing.

Not VS Code, and not Notepad++. Both load the entire file into memory as text, and both will do it, just slowly, with a warning, and no useful table view at the end.

The file is on your phone

The 1GB case is hard here: phones have less RAM than laptops and stricter limits on what an app may hold. Nothing on a phone opens a full gigabyte comfortably, so test any app claiming otherwise before you trust it. What phone apps do well is the range just below: the 100–500MB, several-hundred-thousand-row exports that a phone spreadsheet silently truncates.

CSV File Viewer - Smart CSViPhone & iPad · Android

Opens large files without truncating them, then answers questions about the data with SQL or in plain language, which is usually what you wanted from a file too big to scroll. iOS and Android.

What it does

Before trusting any phone app with a file that matters, run the check in the last section of this page.

You need to edit it and save it back

This is the case with the fewest honest answers, because "edit" means holding the file and writing it out again.

  • A few surgical changes — fix a value, drop a column: do it with DuckDB or awk and write a new file. It is faster than any interactive tool and it leaves the original intact.
  • Real interactive editing on Windows: EmEditor.
  • Real interactive editing elsewhere: split the file first. split -l 1000000 big.csv part_ gives you pieces a spreadsheet can hold, at the cost of having to reassemble them (cat works, but delete the repeated header rows).
  • Under about 500,000 rows, a phone editor handles it directly:
CSV Editor - Smart CSViPhone & iPad

Spreadsheet-style editing on 500,000-row files with every row written back on save, the free version included. iOS and iPadOS.

What it does

Converting the problem away

If you'll touch the file more than twice, stop treating it as a CSV:

duckdb -c "COPY (SELECT * FROM 'big.csv') TO 'big.parquet'"

Parquet is columnar, typed and compressed. The 1GB CSV becomes 100–200MB, queries that scan a few columns get five to twenty times faster, and the types you fought over are now stored in the file. DuckDB, Polars and pandas all read it natively. This is the single highest-value ten seconds in this article.

Testing whether a tool actually opened it

Plenty of tools open a large CSV by opening part of it and not mentioning which part. Before you draw a conclusion from a file, check:

  1. Count the rows first, outside the tool: wc -l big.csv. Subtract one for the header.
  2. Go to the last row in the tool and compare the count.
  3. Check the last row's contents against tail -1 big.csv.

If those disagree, the tool truncated. Any conclusion you draw (a total, a max, "there are no orders from that region") is now wrong in a way nothing on screen tells you. Silent truncation is the actual danger with large files, more than crashes are: a crash is at least honest.

If it's specifically Excel that failed

Two neighbouring problems that are easy to confuse. If Excel refused the file or showed only part of it, that's the row ceiling, and how to open a large CSV file that Excel won't open covers the routes around it. If Excel opened the file and then froze, hung or died, that's a different fault with different causes, covered in why Excel crashes on large files.

Questions

How many rows are in a 1GB CSV file?
Roughly five to ten million for ordinary tabular data, depending on how many columns there are and how long the values run. That is already five times an Excel worksheet's 1,048,576-row limit, which is why the file size matters less than the row count.
What is the fastest way to open a 1GB CSV file?
DuckDB, if you want an answer out of it. It queries the file in place in about a second and uses a few hundred megabytes of RAM. If you want to scroll through it, EmEditor on Windows opens it as a grid, and `less -S` works anywhere and starts instantly.
Can Excel open a 1GB CSV file?
Not directly. A worksheet holds 1,048,576 rows and a 1GB CSV usually has several times that, so Excel loads what fits and stops. Power Query can process the whole file as a load-and-summarise step without putting every row on a sheet, but that is a different workflow from opening it.
Why does pandas use 8GB of RAM to read a 1GB CSV?
Because text columns are stored as Python objects, each with its own memory overhead, and read_csv builds the whole frame in memory before returning. Pass a dtype map and a chunksize, use the pyarrow engine, or use Polars instead, whose scan_csv plans the query lazily and reads far less.
How do I know whether a tool truncated my CSV?
Count the rows outside the tool with `wc -l`, then jump to the last row inside the tool and compare, including the last row's contents against `tail -1`. Silent truncation is more dangerous than a crash, because every total and every maximum you read off the screen is then quietly wrong.
Should I convert a large CSV to Parquet?
If you will query it more than twice, yes. One DuckDB COPY command turns a 1GB CSV into a 100–200MB Parquet file that is typed, columnar and five to twenty times faster to scan, and DuckDB, Polars and pandas all read it natively.

uos Builds CSV Editor and Smart CSV Viewer, and benchmarked most of this table out of self-interest.