Our you can do it manual in Excel.
1. Select the range of cells, or make sure that the active cell is in a table.
2. On the Data tab, in the Data Tools group, click Remove Duplicates.
3. Do one or more of the following:
* Under Columns, select one or more columns.
* To quickly select all columns, click Select All.
* To quickly clear all columns, click Unselect All.
4. Click OK.
(A message is displayed indicating how many duplicate values were removed and how many unique values remain, or if no duplicate values were removed.)
5. Click OK.
You can easily make a program that does this, or do it manual if it is to hard 🙂