I am trying to remove all html-elements (tags) that contain a specific text-string. I have 2376 html-documents, all with different doctype standards. Some even don't have a doctype (might be irrelevant to this question).
So, I am looking for a text string that says "How to cite this paper", and I've found that it is enclosed within either a <p>-tag
, <h4>-tag
or a <legend>-tag
.
The <p>-tag
often looks like this,
<p style="text-align : center; color : Red; font-weight : bold;">How to cite this paper:</i></p>
The <h4>-tag
often looks like this,
<h4>How to cite this paper:</h4>Antunes, P., Costa, C.J. & Pino, J.A. (2006).
The <legend>-tag
looks like this,
<legend style="color: white; background-color: maroon; font-size: medium; padding: .1ex .5ex; border-right: 1px solid navy; border-bottom: 1px solid navy; font-weight: bold;">How to cite this paper</legend>
The task at hand is to find these tags and remove them from the file, and then save the file again. I do have more tags to remove, but need some help understanding HAP and XPath, and how to locate specific tags based on their values or other unique data.
So far I have come up with this code in C#, it's a console application. This is my Main (sorry for bad indentation),
//Variables
string Ext = "*.html";
string folder = @"D:\websites\dev.openjournal.tld\public\arkivet\";
IEnumerable<string> files = GetHTMLFiles(folder, Ext);
List<string> cite_files = new List<string>();
var doc = new HtmlDocument();
//Loop to match all html-elements to query
foreach (var file in files)
{
try
{
doc.Load(file);
cite_files.Add(doc.DocumentNode.SelectNodes("//h4[contains(., 'How to cite this paper')]").ToString());
cite_files.Add(doc.DocumentNode.SelectNodes("//p[contains(., 'How to cite this paper')]").ToString());
}
catch (Exception Ex)
{
Console.WriteLine(Ex.Message);
}
}
//Counts numbers of hits and prints data to user
int filecount = files.Count();
int citations = cite_files.Count();
Console.WriteLine("Number of files scanned: " + filecount);
Console.WriteLine("Number of citations: {0}", citations);
// Program end
Console.WriteLine("Press any key to close program....");
Console.ReadKey();
And this is the private method that looks through directories for files,
//List all HTML-files recursively and return them to a list
public static IEnumerable<string> GetHTMLFiles(string directory, string Ext)
{
List<string> files = new List<string>();
try
{
files.AddRange(Directory.GetFiles(directory, Ext, SearchOption.AllDirectories));
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return files;
}
The unique thing seems to be "How to cite this paper" so I am trying to find all specific tags that contain these exact words, and then remove them. My Notepad shows that there should be 1094 files with this phrase, so I am trying to get them all. :)
Any help greatly appreciated! :)