How to remove duplicate lines
- Paste your list with one item on each line — email addresses, keywords, product codes, names or URLs.
- Pick your options. Decide whether Apple and apple count as the same item, whether spaces at the ends of lines should be ignored, and whether blank lines should be dropped.
- Copy or download the cleaned list. The note under the result tells you how many lines were removed.
What this tool does
It reads your text line by line and keeps only the first occurrence of each line. Later repeats are removed, and everything else stays in its original order. That last point matters: many spreadsheet "remove duplicates" features sort the data at the same time, which destroys an order you may have spent time on — a priority list, a sequence of steps, or a ranking. If you do want the list sorted, run the result through the Text Sorter afterwards.
How the options work
- Case-sensitive (on by default): London and london are treated as different lines. Turn it off to treat them as duplicates — the version that appears first is the one kept, with its original capitalisation.
- Trim spaces at line ends: ignores invisible spaces and tabs at the start and end of each line. Text copied from spreadsheets and PDFs very often contains these, and they are the most common reason two lines that look identical are not treated as duplicates.
- Remove empty lines: drops blank lines entirely. When this is off, the first blank line is kept and later blank lines are treated as duplicates of it.
Examples
Input:
anna@example.com
ben@example.com
Anna@example.com
anna@example.com
chris@example.comWith Case-sensitive on, one line is removed (the exact repeat of anna@example.com), leaving four. With it off, two lines are removed, because Anna@example.com is also treated as a repeat — leaving anna@example.com, ben@example.com and chris@example.com. For email addresses, case-insensitive matching is usually what you want, since mail providers treat addresses case-insensitively in practice.
Paste two or three keyword exports one after another, turn on Trim and turn off Case-sensitive, and you get a single de-duplicated master list in the order the keywords first appeared.
Tips and best practices
- Normalise first if needed. If your list mixes "Street" and "St.", the tool cannot know they mean the same thing — only identical lines are merged. A quick pass with the Case Converter can help standardise capitalisation.
- Fix spacing inside lines. "New York" with two spaces is different from "New York". Run the list through Remove Extra Spaces to collapse repeated spaces inside lines.
- Keep a copy of the original if the list is important. The tool never changes your input box, but it is good practice when cleaning data.
- Count what's left. Paste the result into the Word Counter to see the number of lines as paragraphs.
Frequently asked questions
Which copy of a duplicate is kept?
The first one. Every later line that matches it is removed, and the order of the remaining lines does not change.
Why were two identical-looking lines not removed?
Usually because of an invisible difference: a trailing space, a tab, a non-breaking space, or different capitalisation. Turn on Trim spaces and turn off Case-sensitive, and clean internal spacing with Remove Extra Spaces if necessary.
Can I remove duplicates from a CSV file?
Yes, if each row is on its own line — each whole row is compared. The tool does not compare individual columns, so two rows that share an email but differ in another column are both kept.
Is my list uploaded anywhere?
No. The comparison runs in your browser, so customer lists and other private data never reach our server.
Last updated . Spotted a mistake? Let us know and we will correct it.