Remove Duplicate Lines
Paste a list with one item per line and every repeated line is removed as you type, leaving the first copy of each in its original order. You can ignore capitalisation, drop blank lines, and see exactly which lines were removed. Free, no sign-up, and the text never leaves your browser.
Options
How to Use the Duplicate Line Remover
- Paste your list. One item per line: names, email addresses, keywords, reference entries, URLs or rows of a data export.
- Set the options. Tick Case-insensitive if "Apple" and "apple" should count as the same line. Tick Also remove blank lines to delete empty lines as well as repeats.
- Read the cards. Total Lines, Unique Lines and Duplicates Removed update as you type.
- Copy the output. Cleaned Output holds the deduplicated list. Below it, Removed Duplicates shows each repeated line once (the first 20, then a count of the rest), so you can check nothing was dropped by mistake.
How Lines Are Compared
The tool reads your text from the top. The first time it meets a line, it keeps it; every later line that matches is removed. That means the list never reorders — the survivors appear in the sequence they first appeared in — and it is why this is called a first-occurrence method. If you want the result alphabetised as well, run the output through Sort Lines Alphabetically.
Matching is exact, character for character. By default the comparison is case-sensitive, and a line that differs only by a trailing space counts as a different line. The tool does not trim spaces, so "[email protected]" and "[email protected] " with a stray space at the end are both kept. If a copied list has invisible spaces, clean it with the Extra Space Remover first.
A Worked Example
Take this nine-line list, which includes two blank lines and mixed capitalisation:
Input
apple Banana apple banana cherry Apple banana
| Options | Unique lines | Removed | Output |
|---|---|---|---|
| Defaults (case-sensitive) | 6 | 3 | apple, Banana, banana, (blank), cherry, Apple |
| Case-insensitive | 4 | 5 | apple, Banana, (blank), cherry |
| Case-insensitive + remove blank lines | 3 | 4 | apple, Banana, cherry |
Notice what the blank-line option does to the counts. With it on, the two empty lines are dropped before counting, so Total Lines falls from 9 to 7. Without it, a blank line is just another line: the first one is kept and the second is removed as a duplicate. Notice also that the survivor keeps its own spelling — "Banana" stays as written even though "banana" matched it.
What People Use It For
- Email and contact lists. Remove people who appear twice after merging two exports.
- Keyword and tag lists. Clean a long list of search terms before you import it.
- Reference lists. Strip duplicate entries when you have pasted references from several sources into one list.
- Data and logs. Pull out the distinct values from a single column or a log extract.
Which Tool Do You Need?
| If you want… | Use |
|---|---|
| Repeated lines deleted, order kept | This page |
| Repeated words deleted, or "the the" typos fixed | Duplicate Word Remover |
| Every distinct word in a text as an alphabetised list | Unique Word Extractor |
| The items two lists share, or that only one has | List Comparison Tool |
| Two lists joined into one list | List Merger |
| Only the lines that contain a word or pattern | Line Filter |
Frequently Asked Questions
- How do I remove duplicate lines from text?
- Paste the text into the box, one item per line. The cleaned list appears straight away under Cleaned Output with the repeats removed, and you can copy it with one click. There is no button to press.
- Does removing duplicate lines keep the original order?
- Yes. The first time a line appears it is kept in place, and every later identical line is deleted. The remaining lines stay in the order they first appeared, so nothing is reshuffled.
- Is the comparison case-sensitive?
- By default it is, so "Apple" and "apple" are two different lines. Tick Case-insensitive to treat them as duplicates. The first one keeps its original capitalisation either way.
- Why are two lines that look the same not removed?
- Usually one has a trailing space, a tab or a different capital letter. The match is exact and the tool does not trim lines, so "item" and "item " differ. Turn on Case-insensitive for capitals, and clean stray spaces before pasting.
- Can I remove blank lines at the same time?
- Yes, tick Also remove blank lines. Without it, one blank line is kept, because the first blank line counts as a line and only the later empty lines are removed as duplicates. With the option on, a line that holds only spaces counts as blank.
- How do I deduplicate and sort a list?
- Remove the duplicates here, copy the output, and paste it into Sort Lines Alphabetically. Either order gives the same set of lines, but deduplicating first leaves less to sort.
- Is my list uploaded?
- No. Lines are compared in your browser and the text is not sent or stored anywhere.