2

I wrote a simple scraper that takes a URL and scrapes just one Xpath from that document. In the local machine, it works perfectly fine, but after building as a docker image and running up in a docker container, it returns a null value. It's because it can not load that HTML page anymore.

var web = new HtmlWeb();
            var doc = web.Load(_url);
            var HeaderNames = doc.DocumentNode.SelectNodes("//*[@id='productTitle']").FirstOrDefault();
            return HeaderNames.InnerText.Trim();

I'm using HTML-agility-pack if it does matter. How can I solve this problem? or has anyone had a similar problem?

also my docker image

    FROM mcr.microsoft.com/dotnet/aspnet:6.0-focal AS base
WORKDIR /app
EXPOSE 8080

ENV ASPNETCORE_URLS=http://+:8080
RUN adduser -u 5678 --disabled-password --gecos "" appuser && chown -R appuser /app
USER appuser

FROM mcr.microsoft.com/dotnet/sdk:6.0-focal AS build
WORKDIR /src
COPY ["Scraper.csproj", "./"]
RUN dotnet restore "Scraper.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "Scraper.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "Scraper.csproj" -c Release -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Scraper.dll"]
2
  • 1
    Could you ping your url address from container?
    – D-Shih
    Apr 5, 2022 at 6:31
  • 1
    @D-Shih - no I can't, although in local I can ping any address Apr 5, 2022 at 6:48

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.