How do you read a very large file line by line in Node.js?
Create a read stream and pass it to readline.createInterface, then loop with for await. Only a small chunk sits in memory at a time, unlike readFile, which loads the whole file.
Explain it simply
Reading a 2 GB log file with readFile is like trying to photocopy a thick book by laying all pages on the floor first. A stream lets you read a few pages, process them, and move on. The readline module sits on top of the stream and cuts the incoming bytes into lines for you.
Example
const fs = require("fs");
const readline = require("readline");
async function countErrors(file) {
const rl = readline.createInterface({
input: fs.createReadStream(file),
crlfDelay: Infinity, // treat \r\n as a single line break
});
let errors = 0;
for await (const line of rl) {
if (line.includes("ERROR")) errors++;
}
return errors;
}
countErrors("server.log").then((n) => console.log(n, "errors"));Memory use stays roughly constant whatever the file size, because the stream reads in chunks (64 KB by default for files) and readline hands you one line at a time.
The for await loop also respects backpressure: the stream pauses while your loop body is busy. The same pattern suits CSV imports, such as loading a big list of student records into a database.
What to say in the interview
"I wrap fs.createReadStream in readline.createInterface and iterate with for await, so memory stays small. readFile would load the entire file at once and could crash on big inputs."
Common mistakes
- Calling
readFileand then.split("\n")on a multi-gigabyte file. - Forgetting
crlfDelay: Infinityand getting empty lines on Windows files.