0

I need to remove duplicate records (just to keep one copy) from a MySQL table in MyISAM format. I have read many questions in Stackoverflow about similar issues but always the people use an ID field as unique id or similar and I haven't this field because my "Key" could be duplicated. I want to remove rows with the 'Fecha' and 'Equip' duplicated.

CREATE TABLE `pfc` (
`Fecha` datetime NOT NULL,
`Equip` int(10) unsigned NOT NULL,
`Value` double NOT NULL,
KEY `Fecha` (`Fecha`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AVG_ROW_LENGTH=21 ROW_FORMAT=FIXED;

An example table data:

Fecha              | Equip | Value 
06/02/2011 0:00:11 |  22   |  29.0
06/02/2011 0:22:11 |  22   |  45.3
06/02/2011 0:00:11 |  22   |  29.0

The result should be:

Fecha              | Equip | Value 
06/02/2011 0:00:11 |  22   |  29.0
06/02/2011 0:22:11 |  22   |  45.3

This structure is not possible to change. I cannot use PRIMARY KEY or UNIQUE INDEX solutions. To create a temporal table without duplicates and then rename would be a poor solutions also, because the KEY and another parameters will be lost.

Thanks

8
  • 1
    Why would a temporary table be a poor solution? You can always recreate indexes. Also, if Equip and Fecha are duplicates, what value should be selected if the values differ?
    – Jacob
    Aug 18, 2011 at 11:39
  • Temporary tables could be work but not to create a table "new_fc" as "Octopus-Paul" and then rename "new_pfc" to "pfc". How can I use temporaty tables to solve the issue?
    – Jaime M.
    Aug 18, 2011 at 12:05
  • You don't have to use temporary tables, there's more elegant solution to the problem.
    – N.B.
    Aug 18, 2011 at 12:09
  • Without temp. table it is not possible. Your table does not have id and MySQL does not support internal row id. The table contains two identical rows - '06/02/2011 0:00:11|22|29.0'. How to specify the line that should be removed?
    – Devart
    Aug 18, 2011 at 12:32
  • @Devart - do you have any source that MySQL doesn't support internal row ids? FYI, it's not up to MySQL but storage engine used.
    – N.B.
    Aug 18, 2011 at 12:39

3 Answers 3

9
ALTER IGNORE TABLE `pfc` ADD UNIQUE(`Fetcha`, `Equip`);

That will keep the first record it finds and remove duplicates from your table.

From MySQL manual:

IGNORE is a MySQL extension to standard SQL. It controls how ALTER TABLE works if there are duplicates on unique keys in the new table or if warnings occur when strict mode is enabled. If IGNORE is not specified, the copy is aborted and rolled back if duplicate-key errors occur. If IGNORE is specified, only the first row is used of rows with duplicates on a unique key. The other conflicting rows are deleted. Incorrect values are truncated to the closest matching acceptable value.

Edit:

To create an exact copy of your table, use the following:

CREATE TABLE table_copy SELECT * FROM pfc;

That will copy the table structure and the data. After that, run the ALTER command to add the UNIQUE constraint and filter out the records that are duplicated.

11
  • Sorry, as I've said, I cannot use UNIQUE because the format table is untouchable.
    – Jaime M.
    Aug 18, 2011 at 12:24
  • So create a copy of the table, add the constraint, filter the results, truncate the original table and populate it with the contents of the copied table. It's about 2 minutes of work in total. You can TRUNCATE or DELETE FROM, right?
    – N.B.
    Aug 18, 2011 at 12:28
  • I would create a copy of the table if the format is identical but I don't know how to do that.
    – Jaime M.
    Aug 18, 2011 at 12:34
  • Sorry but the copy is not exact. I get a table like this: CREATE TABLE pfc_new` ( Fecha datetime NOT NULL, Equip int(10) unsigned NOT NULL, Value double NOT NULL, ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
    – Jaime M.
    Aug 18, 2011 at 12:45
  • So you're saying that a computer, after being instructed to create an exact copy of the target table, failed to do so and it did something else? Now how can that be? :)
    – N.B.
    Aug 18, 2011 at 12:50
0

Just as a test, try this to see if it is what you want

CREATE TABLE new_pfc AS
SELECT * FROM pfc GROUP BY fecha, equip
1
  • As I've said, to create a temporal "new_pfc" is a poor solution because the index and another formats are lost. I need to avoid change the table format.
    – Jaime M.
    Aug 18, 2011 at 12:08
0
DELETE n1 FROM pfc n1, pfc n2 WHERE n1.Fecha = n2.Fecha AND n1.Equip=n2.Equip AND n1.Value=n2.Value

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.