Remove Duplicate Lines

Remove duplicate lines from text while keeping the original order. Supports case-sensitive and case-insensitive modes.

Remove Duplicate Lines tool

Loading...

Example

Input: 5 lines: apple, banana, apple, cherry, banana

Result: apple / banana / cherry (3 lines)

Each line is kept the first time it appears and later repeats are dropped, so the original order is preserved.

  1. Paste your text containing duplicate lines into the input area.
  2. Leave the Case Sensitive toggle on for exact matching, or turn it off so lines differing only in letter case count as duplicates.
  3. Optionally enable the Trim Whitespace option to ignore leading and trailing spaces during comparison.
  4. Review the deduplicated output, which keeps the first occurrence of each line in its original order.
  5. Check the stats bar under the input for the original line count.
  6. Click the copy button to copy the cleaned text to your clipboard.

Removing Duplicate Lines Without Losing Order

This tool deletes repeated lines from a list and keeps the first copy of each one in its original place. Paste 500 email addresses that include 60 repeats and you get 440 unique lines back, in the order you pasted them.

Two switches decide what counts as a repeat. Case Sensitive is on by default, so "Apple" and "apple" are kept as two lines. Turn it off to treat them as the same. Trim Whitespace is off by default. Turn it on to ignore spaces at the start and end of a line, which often sneak in when you copy from a spreadsheet or PDF. Either way, the line that stays is the original, exactly as you typed it. Blank lines are treated like any other line, so only the first empty line survives. The stats bar under the input shows how many lines you started with. Compare it with the output to see how many were removed.

Typical jobs include cleaning mailing lists, merging keyword lists, dropping repeated log entries and tidying exported data. To put the unique lines in order afterwards, use Sort Lines. To turn them into a comma-separated or SQL list, use the List Converter.

Frequently Asked Questions

No. The tool reads your text from top to bottom and keeps each line the first time it appears. Later copies are dropped, and nothing is moved. If your list was in date order or ranked by importance, it stays that way. Sort the result separately if you want it in alphabetical order.
Turn off Case Sensitive. Then "London", "london" and "LONDON" count as the same line, and only the first one is kept, with its capital letters as you typed them. If you want every line in the same case as well, run the result through a case converter afterwards.
This tool keeps the first blank line, because it treats empty lines like any other line. To delete every blank line, paste the result into the Whitespace Remover and turn on Remove Blank Lines. That also removes lines that contain only spaces.
Not directly, because this tool works on lines. First put one value on each line, for example with the List Converter set to Newline separated output. Remove the duplicates here, then convert the list back to commas. Watch for spaces after commas, or turn on Trim Whitespace so they do not create false differences.

Related Tools

Sort Lines

Sort lines of text alphabetically (A-Z or Z-A) with natural number order, by line length, or shuffle and reverse them.

Text and Writing

Word Counter and Character Counter

Count words, characters, sentences, and paragraphs in your text. Includes reading time and speaking time estimates.

Text and Writing

Whitespace Remover and Trimmer

Remove extra whitespace, trim leading and trailing spaces, collapse multiple spaces, and clean up messy text formatting.

Text and Writing

Find and Replace

Find and replace text with regular expressions, capture groups, case-sensitive matching, and a live match count.

Text and Writing

Text Diff and Compare Tool

Compare two blocks of text side by side and highlight the differences. Find additions, deletions, and changes instantly.

Text and Writing

List Converter

Transform lists between formats - comma-separated, newline, bulleted, numbered, JSON array, SQL IN clause, and HTML.

Text and Writing