TodayTools

Remove duplicate lines

Paste a list and keep one copy of every line. Ctrl+C copies the result.

PrivateNo uploadsRuns in your browser

Donate
Try an exampleTry:

Recipe

Duplicate lines

Keep
Paste text to see what each step removes.
Show

How to remove duplicate lines

  1. Paste your list

    Paste one item per line, or open a text or CSV file. Big files are read in the background.

  2. Choose how lines match

    Tick Ignore case to treat Milk and milk as the same, and Ignore spaces for lines that differ only by spaces. Choose to keep the first or the last copy.

  3. Copy the list

    The counter shows how many repeats were removed, and the step lists the lines that were repeated most. Copy or download the clean list.

Good to know

  • The order of your list stays the same. Add a Sort step after this one if you want it A to Z too.
  • Blank lines are kept by default so paragraphs stay apart. Untick Keep blank lines to remove repeated blank lines as well.
  • Keep last is handy for logs and exports, where the newest copy of a line is at the bottom.
  • Big files are fine. Drop a text file of 50 MB or more: the work happens in the background, the page stays smooth, and Download saves the full result.

Where people use it

  • Mailing list

    Remove repeated emails before you send an invite so no one gets it twice.

  • Class or team list

    Merge names from many sheets and keep one copy of each.

  • Keywords or tags

    Clean a long list of tags or search words.

  • Sort what is left

    Put the unique lines in A to Z order.

    Sort lines

  • More clean-up

    Need to fix spaces or blank lines too? Add steps in the full text cleaner.

    Text cleaner

Why this one

It tells you what was removed and which lines repeated most, not just the clean list.

  • Shows how many repeats were removed.
  • Lists the most repeated lines.
  • Handles long lists on your own device.

What you can change

  • Keep the first or last copy
  • Ignore upper and lower case
  • Ignore extra spaces

People find this page by searching

  • remove duplicate lines
  • remove duplicates from list
  • delete duplicate lines online
  • dedupe list
  • unique lines online
  • remove duplicate emails from list
  • remove duplicate names
Cite this tool
  • APA
    sdev.in. (n.d.). Remove duplicate lines online [Web tool]. https://www.sdev.in/tools/remove-duplicate-lines
  • MLA
    "Remove duplicate lines online." sdev.in, https://www.sdev.in/tools/remove-duplicate-lines.
  • Chicago
    sdev.in. "Remove duplicate lines online." https://www.sdev.in/tools/remove-duplicate-lines.
  • BibTeX
    @misc{sdev-remove-duplicate-lines,
  title = {Remove duplicate lines online},
  author = {{sdev.in}},
  howpublished = {\url{https://www.sdev.in/tools/remove-duplicate-lines}},
  note = {Web tool}
}

Questions people ask

Does it keep the order of my lines?

Yes. Each line stays where its first copy was (or its last copy, if you choose Keep last). Nothing is sorted unless you add a Sort step.

Can it ignore upper and lower case?

Yes. Tick Ignore case and lines such as Milk, MILK and milk count as one. The copy that is kept is shown exactly as you typed it.

How do I see how many duplicates were removed?

The step shows the number of lines removed, the counter under the boxes shows lines before and after, and the most repeated lines are listed with how many copies there were.

Can I remove duplicate words instead of lines?

Add the Join or split step before it and choose Split into lines with Space as the separator. Each word goes on its own line, then the duplicate step removes the repeats.

Is my text uploaded anywhere?

No. Every step runs in your browser, on your own device. Nothing you paste or open is sent to a server or kept anywhere except this page.

Donate