Line Stream Demo

This is a demo project for the line-by-line Node JS library. It opens a given text file and displays the contents line by line, as the name would suggest.

The fs.readFile function in the default File System library is a very easy, convenient way to open and read files. However, the catch is that it reads the file all at once and stores the contents in memory. This is fine for smaller files but it becomes an issue for files with a large or unknown size as attempting to store the contents of large files in memory can cause fatal errors. Therefore, the file should be read in smaller chunks rather than as a whole. There is no right or wrong way to go about this but it depends on the project and your needs. If the target needs to be read per-line, I wholeheartedly recommend this library.

I have provided an example text file which simulates a chat log between two people. Each line corresponds to a message. The file's size is only ~8kb so it would be safe to read all at once. However, because only one line is stored in memory at a time, the actual size does not matter.

---

Usage

1. Open a terminal in the project folder.
2. Run npm install to download the 'line-by-line' package.
3. Run node index to execute the demo.

---

Message Format

[yyyy-mm-dd HH:MM:SS] User : Hello World
[2021-01-13 10:01:34] Alice : Lorem ipsum

---

Credits

• @tjohnston-softdev - Demo project.
• @Osterjour - Library developer.
• Chat messages generated with Lorem Ipsum IO
• Timestamp generation assisted by RANDOM.ORG

---

Further Reading

• Reading a File Line by Line in Node.js
  -Vasyl Lagutin

• Reading files with Node
  -Flavio Copes

• Using Node.js to Read Really, Really Large Datasets & Files
  -Paige Niedringhaus

• Node.js streams for handling huge files
  -Krishna Singh

• Reading Large Structured Text Files in Node.js
  -Momtchil Momtchev

• Node.js | fs.createReadStream() Method
  -Geeks For Geeks

Links Retrieved: 1 April 2021

---

Disclaimer

This demo project is licensed under CC0 1.0 Universal. I only wrote this as a reference for using the 'line-by-line' library to read large files. This is not an official demo and I am in no way affiliated with @Osterjour or any of his contributors. While I will make an effort to maintain this repository where applicable, I do not accept any responsibility for how you use this code or 'line-by-line' as a whole.