12

I have 2 Excel spreadsheets I am trying to compare:

$OleDbAdapter = New-Object System.Data.OleDb.OleDbDataAdapter “Select * from [Report$]“,”Provider=Microsoft.ACE.OLEDB.12.0;Data Source=S:\FIS-BIC Reporting\Report Output Files\Product-Marketing\TEST_XI\ECM - Pipeline by LOB_04182013_040544.xls;Extended Properties=”"Excel 12.0 Xml;HDR=YES”";”
$RowsReturned = $OleDbAdapter.Fill($DataTable)

$OleDbAdapter2 = New-Object System.Data.OleDb.OleDbDataAdapter “Select * from [Report$]“,”Provider=Microsoft.ACE.OLEDB.12.0;Data Source=S:\FIS-BIC Reporting\Report Output Files\Product-Marketing\ECM - Pipeline by LOB_04182013_074004.xls;Extended Properties=”"Excel 12.0 Xml;HDR=YES”";”
$RowsReturned2 = $OleDbAdapter2.Fill($DataTable2)

Compare-Object $DataTable $DataTable2 

It returns nothing. I know that in the 6th column, they are different. If I specify "-property F6", it does return the difference. Any idea why it doesn't unless I specify the property? The number of columns can vary (though will be the same for each of the files in the comparison), so specifying the properties specifically won't work.

0

2 Answers 2

31

If you don't specify the -Property parameter, Compare-Object doesn't compare all properties, it compares the results of invoking the .ToString() method on both objects. So, Compare-Object $DataTable $DataTable2 compares $DataTable1.ToString() with $DataTable1.ToString(). The .ToString() method returns an empty string when invoked on a DataTable object, so there is no difference to report.

For example:

$file1 = Get-Item somefilename
$file1 = Get-Item anotherfilename
Compare-Object $file1 $file2

This will return the difference between the full paths of the two files, like this:

InputObject              SideIndicator
-----------              -------------
<path>\anotherfilename   =>
<path>\somefilename      <=

That's because invoking .ToString() on a FileInfo object returns its FullName property, so you're comparing the files' full path names.

Although the -Property parameter accepts multiple properties, listing all the properties is not the solution. Aside from being very tedious, it will not give you the results you want. If you list multiple properties, Compare-Object compares the combination of all the properties, and if any one of the listed properties is different, it returns a result showing all the listed properties (both ones that are the same and ones that are different) as a single difference.

What you need to do is iterate over a list of properties, and invoke Compare-Object once for each property:

$properties = ($DataTable | Get-Member -MemberType Property | Select-Object -ExpandProperty Name)
foreach ($property in $properties) {
  Compare-Object $DataTable $DataTable2 -Property "$property" | Format-Table -AutoSize
}
  • In most cases, when comparing all properties of two objects, you'd want to use Get-Member -MemberType Properties, in order to get cover all property types. However, if you're comparing DataTable objects, you're better off using Get-Member -MemberType Property so that you're comparing only the properties corresponding to data fields, not other properties of the DataTable objects that have nothing to do with the data.

  • This is written assuming that the number of columns is the same, as you stated, or at least that the number of columns in $DataTable2 doesn't exceed the number of columns in $DataTable.

    If you can't reliably assume that, derive the $properties array from whichever one has more columns, by comparing ($DataTable | Get-Member -MemberType Property).Count with ($DataTable2 | Get-Member -MemberType Property).Count and using the properties from whichever is greater.

  • Using Format-Table is important, it's not just there to make things look pretty. If you list multiple objects of the same type (in this case, arrays), PowerShell remembers the format of the first object, and uses it for all subsequent objects, unless you explicitly specify a format. Since the name of the first column will be different for each property (i.e., each column from the spreadsheet), the first column will be empty for all but the first difference encountered.

    The -AutoSize switch is optional. That is there just to make things look pretty. But you must pipe the results to a formatting filter. You can also use Format-List if you prefer.

4
  • Thanks for the thorough response. I didn't know it didn't compare all properties. I am havign a problem though. I really want to do a row by row compare. To do that, I am thinking I need to add a key called something like RowNum to each and specify that as the key. Aug 22, 2013 at 16:01
  • Are you saying that you want to compare entire rows, and return results if any of the columns in the compared rows contain different data? If so, see the part in the middle where I wrote "Although the -Property parameter accepts multiple properties, listing all the properties...will not give you the results you want." If you want to compare entire rows, listing all the properties (i.e., column names) in the -Property parameter will give the results you want. Is that the idea?
    – Adi Inbar
    Aug 22, 2013 at 16:28
  • Correct - entire rows. It would be nice to know what column had the difference, but not essential. Is there a way to do that though without listing each property? Each spreadsheet can have a different number of columns. I am planning on doing a bulk compare of 1000s of spreadsheets (they are sets, so only comparing spreadsheet a to b, c to d, etc). A and B have the same # of columns, but c and d will be different. Aug 22, 2013 at 20:22
  • You could read all the property names of $DataTable1 into an array and use that as the -Property argument. I'd need to know more about the structure of these DataTables in order to be more specific about how. I don't have an OLEDB driver for Excel installed, so I can't reproduce this locally. What are the property names like? You say that you can use F6 as a property name, but that doesn't make sense to me. Wouldn't the property names be the same as the Excel column names, i.e. A, B, C, D, etc? How can the row # be part of the property names if all rows have the same properties?
    – Adi Inbar
    Sep 4, 2013 at 1:18
8

Adi Inbar's helpful answer contains good background information about how Compare-Object works.

However, there is a way to use -Property generically to compare all column values - assuming that both input tables have the same column structure, or that the the tables should only be compared by the first table's columns:

# The original collection.
$coll1 = [pscustomobject] @{ one = '1a'; two = '2a'; three = '3a' },
         [pscustomobject] @{ one = "1b"; two = "2b"; three = '3b' }

# The other collection to compare the original to.
# Note the difference in the 2nd object in column 'two'
$coll2 = [pscustomobject] @{ one = '1a'; two = '2a'; three = '3a' },
         [pscustomobject] @{ one = "1b"; two = "2b!"; three = '3b' }

# PSv3+: Get the array of all property names to compare 
#        from the original collection.
# Note: 
#      * The assumption is that both collections have the same set of 
#        properties (or that the collections should only be compared by
#        the *first* collection's properties).
#      * In PSv2-, use the following instead:
#         $propsToCompare = $coll1[0].psobject.properties | % { $_.name }
$propsToCompare = $coll1[0].psobject.properties.name

# Compare the 2 collections by all property values.
# -PassThru means that any input object in which a difference is found
# is passed through as-is.
Compare-Object $coll1 $coll2 -Property $propsToCompare -PassThru

The above yields:

one two three SideIndicator
--- --- ----- -------------
1b  2b! 3b    =>           
1b  2b  3b    <=           

Note how => tells you that the selected object is exclusive to the right side and vice versa for <=

A caveat is that Compare-Object is slow, because it cannot make assumptions about input data being sorted and therefore has to read and compare both input collections in full.

With sorted input, you can use -SyncWindow <Int32> to speed things up, but that requires advance knowledge of how many items at most can differ between the two input collections following each difference found, and if the -SyncWindow value is too small, spurious differences will be reported.


If you compare objects as a whole (no -Property argument), PowerShell uses the following comparison method:

Note:

  • Below, LHS refers to an object from the reference collection (-ReferenceObject) and RHS to an object from the difference collection (-DifferenceObject)

  • The short of it is: Unless the types involved implement IComparable (which is true for strings and all primitive .NET types (the CLR's number types and [char])) or have custom .ToString() implementations with instance-specific return values from which equality can be inferred, whole-object comparison will not be meaningful, and objects will be treated as equal even when they aren't.

The following logic is implemented in the TryCompare() engine method, which is used whenever two values must be compared, irrespective of context.

  • If the LHS is a string, string comparison is performed, which is case-insensitive and culture-invariant by default; the -CaseSensitive and -Culture parameters allow you to change that.

  • If both the LHS and the RHS are numbers of (potentially different) .NET primitive types, numeric comparison is performed.

  • Otherwise, an attempt is made to convert the RHS is to the type of the LHS and, if the type supports System.IComparable, its .CompareTo() method is called to determine equality.

  • Otherwise, the LHS and RHS are compared via the Object.Equals() method that all objects inherit or implement or, if the type implements interface IEquatable<T>, IEquatable<T>.Equals().

    • If System.Object.Equals() is called, and a given type doesn't override it, only value types (structs) composed only of other value-type instances will compare meaningfully; for reference types, only two references to the very same object instance are considered equal.

    • Caveat: As of PowerShell Core 7.1.0-preview.2, only if the .Equals() call returns true is the result used. The reason is that the comparison code is also used for ordering (sorting) values, where determining equality alone is not enough. In the context of Compare-Object, not using a false result is actually inappropriate for types that implement IEquatable<T> (and don't also implement IComparable) - see this GitHub issue.

The remaining logic comes from ObjectCommandComparer.Compare() (which is also used by Get-Unique and Select-Object -Unique):

  • If none of the above methods apply, the objects are compared by their .ToString() representations (with the same string-comparison specifics described above).

    • Note that for many types these will not result in meaningful comparison, because the default .ToString() implementation simply returns the full type name (e.g., 'System.IO.FileInfo'), so that all instances of that type compare the same.
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.