21

In my dataset, I have a field which stores text marked up with HTML. The general format is as follows:

<html><head></head><body><p>My text.</p></body></html>

I could attempt to solve the problem by doing the following:

REPLACE(REPLACE(Table.HtmlData, '<html><head></head><body><p>', ''), '</p></body></html>')

However, this is not a strict rule as some of entries break W3C Standards and do not include <head> tags for example. Even worse, there could be missing closing tags. So I would need to include the REPLACE function for each opening and closing tag that could exist.

REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(
    Table.HtmlData,
    '<html>', ''),
    '</html>', ''),
    '<head>', ''),
    '</head>', ''),
    '<body>', ''),
    '</body>', ''),
    '<p>', ''),
    '</p>', '')

I was wondering if there was a better way to accomplish this than using multiple nested REPLACE functions. Unfortunately, the only languages I have available in this environment are SQL and Visual Basic (not .NET).

9
  • As you can see for yourself, the core SQL Server string functions are clumsy at best, ugly at worst, for the sort of problem you are facing. You would have a much easier time IMO doing this using something like Java or .NET, where you could leverage the power of an XML parser. Is there any chance of using a different tool for this problem? Commented Aug 10, 2016 at 8:47
  • Nested REPLACE is much the same as a loop (performance-wise). I would create a 2-dimentional array holding pairs of HTML tags and loop over the string attempting to replace the tags. The advantage is that you can modify (enrich) the array without changing your code.
    – FDavidov
    Commented Aug 10, 2016 at 8:49
  • I totally agree with Tim. Also, what will happen if you have ramenks, javascript or a style block inside your html? you will see them as a part of the content, I doubt that's what you want... Commented Aug 10, 2016 at 8:49
  • I would love to implement it that but this is going into an SSRS report so the best option I have would be Visual Basic... Shudder. There's at least, a guarantee that there will be no JavaScript or CSS.
    – dkmann
    Commented Aug 10, 2016 at 8:50
  • Whichever platform you select (SQL, JAVA ,VBA , FORTRAN...) paradigm I presented above would do the job keeping flexibility.
    – FDavidov
    Commented Aug 10, 2016 at 8:51

8 Answers 8

16
DECLARE @x XML = '<html><head></head><body><p>My text.</p></body></html>'

SELECT t.c.value('.', 'NVARCHAR(MAX)')
FROM @x.nodes('*') t(c)

Update - For strings with unclosed tags:

DECLARE @x NVARCHAR(MAX) = '<html><head></head><body><p>My text.<br>More text.</p></body></html>'

SELECT x.value('.', 'NVARCHAR(MAX)')
FROM (
    SELECT x = CAST(REPLACE(REPLACE(@x, '>', '/>'), '</', '<') AS XML)
) r
4
  • 1
    Thanks @Devart this solves the problem without creating a function and by dealing with unpaired tags. Great solution.
    – dkmann
    Commented Aug 10, 2016 at 10:59
  • Doesn't work if the html contains comments <!-- ... -->
    – Clement
    Commented Sep 14, 2019 at 6:09
  • Need to also replace html comments: REPLACE(REPLACE(REPLACE(@x, '>', '/> '), '</', '<'), '--/>', '-->')
    – Clement
    Commented Sep 14, 2019 at 6:55
  • for the second example, is there a way to put a separator between the text values? i.e. My text.|More text. Commented Apr 25 at 15:33
11

If the HTML is well formed then there's no need to use replace to parse XML.
Just cast or convert it to an XML type and get the value(s).

Here's an example to output the text from all tags:

declare @htmlData nvarchar(100) = '<html>
<head>
</head>
<body>
   <p>My text.</p>
   <p>My other text.</p>
</body>
</html>';

select convert(XML,@htmlData,1).value('.', 'nvarchar(max)');

select cast(@htmlData as XML).value('.', 'nvarchar(max)');

Note that there's a difference in the output of whitespace between cast and convert.

To only get content from a specific node, the XQuery syntax is used. (XQuery is based on the XPath syntax)

For example:

select cast(@htmlData as XML).value('(//body/p/node())[1]', 'nvarchar(max)');

select convert(XML,@htmlData,1).value('(//body/p/node())[1]', 'nvarchar(max)');

Result : My text.

Of course, this still assumes a valid XML.
If for example, a closing tag is missing then this would raise an XML parsing error.

If the HTML isn't well formed as an XML, then one could use PATINDEX & SUBSTRING to get the first p tag. And then cast that to an XML type to get the value.

select cast(SUBSTRING(@htmlData,patindex('%<p>%',@htmlData),patindex('%</p>%',@htmlData) - patindex('%<p>%',@htmlData)+4) as xml).value('.','nvarchar(max)');

or via a funky recursive way:

declare @xmlData nvarchar(100);
WITH Lines(n, x, y) AS (
  SELECT 1, 1, CHARINDEX(char(13), @htmlData)
  UNION ALL
  SELECT n+1, y+1, CHARINDEX(char(13), @htmlData, y+1) FROM Lines
  WHERE y > 0
)
SELECT @xmlData = concat(@xmlData,SUBSTRING(@htmlData,x,IIF(y>0,y-x,8)))
FROM Lines
where PATINDEX('%<p>%</p>%', SUBSTRING(@htmlData,x,IIF(y>0,y-x,10))) > 0
order by n;

select 
@xmlData as xmlData, 
convert(XML,@xmlData,1).value('(/p/node())[1]', 'nvarchar(max)') as FirstP;
4
  • This is great, I didn't know you could call .value and use XPATH. But unfortunately, I do need this to work with unmatched tags.
    – dkmann
    Commented Aug 10, 2016 at 9:18
  • Yeah, it would be awesome if tsql has a standard method to transform even a crappy HTML to the XML type without fail. It's less of a problem if you're the one generating the HTML because then you can make sure the HTML is well formed from the start.
    – LukStorms
    Commented Aug 10, 2016 at 9:33
  • Unfortunately, I have no control over the condition of the HTML it's generated from our ERP system. The user types into a multi-line text field and it is automatically formatted as basic HTML. It's sometimes hard to predict what may come out. Even valid HTML has self closing tags that can be written like: <br>, where this would be invalid XML.
    – dkmann
    Commented Aug 10, 2016 at 9:36
  • Added an alternative that will only extract the first p tag before the cast.
    – LukStorms
    Commented Aug 10, 2016 at 12:33
7

Firstly create a user defined function that strips the HTML out like so:

CREATE FUNCTION [dbo].[udf_StripHTML] (@HTMLText VARCHAR(MAX))
RETURNS VARCHAR(MAX)
AS
     BEGIN
         DECLARE @Start INT;
         DECLARE @End INT;
         DECLARE @Length INT;
         SET @Start = CHARINDEX('<', @HTMLText);
         SET @End = CHARINDEX('>', @HTMLText, CHARINDEX('<', @HTMLText));
         SET @Length = (@End - @Start) + 1;
         WHILE @Start > 0
               AND @End > 0
               AND @Length > 0
             BEGIN
                 SET @HTMLText = STUFF(@HTMLText, @Start, @Length, '');
                 SET @Start = CHARINDEX('<', @HTMLText);
                 SET @End = CHARINDEX('>', @HTMLText, CHARINDEX('<', @HTMLText));
                 SET @Length = (@End - @Start) + 1;
             END;
         RETURN LTRIM(RTRIM(@HTMLText));
     END;
GO

When you're trying to select it:

SELECT dbo.udf_StripHTML([column]) FROM SOMETABLE

This should lead to you avoiding to have to use several nested replace statements.

Credit and further info: http://blog.sqlauthority.com/2007/06/16/sql-server-udf-user-defined-function-to-strip-html-parse-html-no-regular-expression/

1
  • This can be done much easier, without a WHILE loop by using a table of templates to replace (much easier to maintain btw) Commented Aug 12, 2016 at 11:34
1

This is the simplest way.

DECLARE @str VARCHAR(299)

SELECT @str = '<html><head></head><body><p>My text.</p></body></html>'

SELECT cast(@str AS XML).query('.').value('.', 'varchar(200)')
1
  • 1
    Definitely the simplest, but how would I get it to work with unpaired tags?
    – dkmann
    Commented Aug 10, 2016 at 9:19
1

One more solution, just to demonstrate a trick to replace many values of a table (easy to maintain!!!) in one single statement:

--add any replace templates here:

CREATE TABLE ReplaceTags (HTML VARCHAR(100));
INSERT INTO ReplaceTags VALUES
 ('<html>'),('<head>'),('<body>'),('<p>'),('<br>')
,('</html>'),('</head>'),('</body>'),('</p>'),('</br>');
GO

--This function will perform the "trick"

CREATE FUNCTION dbo.DoReplace(@Content VARCHAR(MAX))
RETURNS VARCHAR(MAX)
AS
BEGIN
    SELECT @Content=REPLACE(@Content,HTML,'')
    FROM ReplaceTags;

    RETURN @Content;
END
GO

--All examples I found in your question and in comments

DECLARE @content TABLE(Content VARCHAR(MAX));
INSERT INTO @content VALUES
 ('<html><head></head><body><p>My text.</p></body></html>')
,('<html><head></head><body><p>My text.<br>More text.</p></body></html>')
,('<html><head></head><body><p>My text.<br>More text.</p></body></html>')
,('<html><head></head><body><p>My text.</p></html>');

--this is the actual query

SELECT dbo.DoReplace(Content) FROM @content;
GO

--Clean-Up

DROP FUNCTION dbo.DoReplace;
DROP TABLE ReplaceTags;

UPDATE

If you add a replace-value to the template-table you might even use different values as replacements like replace a <br> with an actual line break...

0
1

SQL Server 2017+

If you have a string splitter function, you can strip HTML tags from virtually any text (well-formed or not):

select string_agg(c.String, null) within group (order by o.Ordinal)
from dbo.SplitString(@Input, N'<') o
    cross apply dbo.SplitString(o.String, N'>') c
where o.Ordinal = 1
    or c.Ordinal = 2;

This will be as performant as your splitter function. It should therefore generally out-perform any of the loop-based solutions.

The replace-based solutions cannot deal with comments or elements that have attributes, which makes them practically useless to me.

Here are my versions of the split and strip functions:

create or alter function dbo.SplitString (
      @String nvarchar(max)
    , @Delimiter nvarchar(4000)
)
returns table with schemabinding
as
    return
    select [key] + 1 as Ordinal, value as String
    from openjson(replace(json_modify(N'[]', N'append $', @String), string_escape(@Delimiter, N'json'), N'","'))

create or alter function dbo.StripHtml (
      @Input nvarchar(max)
)
returns nvarchar(max)
as
begin
    return (
        select string_agg(c.String, null) within group (order by o.Ordinal)
        from dbo.SplitString(@Input, N'<') o
            cross apply dbo.SplitString(o.String, N'>') c
        where o.Ordinal = 1
            or c.Ordinal = 2
    )
end
0

This is just an example. You can use this in script to rmeove any html tags:

 DECLARE @VALUE VARCHAR(MAX),@start INT,@end int,@remove varchar(max)
SET @VALUE='<html itemscope itemtype="http://schema.org/QAPage">
<head>

<title>sql - Converting INT to DATE then using GETDATE on conversion? - Stack Overflow</title>
<html>
</html>
'

set @start=charindex('<',@value)
while @start>0
begin
set @end=charindex('>',@VALUE)

set @remove=substring(@VALUE,@start,@end)
set @value=replace(@value,@remove,'')
set @start=charindex('<',@value)
end
print @value
0

You mention the XML is not always valid, but does it always contain the <p> and </p> tags?

In that case the following would work:

SUBSTRING(Table.HtmlData, 
    CHARINDEX('<p>', Table.HtmlData) + 1, 
    CHARINDEX('</p>', Table.HtmlData) - CHARINDEX('<p>', Table.HtmlData) + 1)

For finding all positions of a <p> within a HTML, there's already a good post here: https://dba.stackexchange.com/questions/41961/how-to-find-all-positions-of-a-string-within-another-string

Alternatively I suggest using Visual Basic, as you mentioned that is also an option.

1
  • 1
    Yes as far as I can see across various tables, it does always seem to contain <p> tags. However, there's a limitation with what you have written if I come across this: <html><head></head><body><p>My text.</p><p>Another paragraph</p></body></html>
    – dkmann
    Commented Aug 10, 2016 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.