4

I am trying to create a t.w.c.BrowserLikePolicyForHTTPS to use as the ContextFactory for a t.w.c.Agent. I am using an internal CA for all the servers I want the Agent to communicate with, so I'd like to be able to tell to load the CA cert (PEM format) and use it as the trustRoot argument to BrowserLikePolicyForHTTPS. I have read the docs and looked at the source, but I have no idea what I am supposed to supply as arguments. I tried providing a PyOPenSSL x509 object, but I get an error:

exceptions.TypeError: ('Could not adapt', <OpenSSL.crypto.X509 object at 0x280b290>, <InterfaceClass twisted.internet._sslverify.IOpenSSLTrustRoot>)

I can see in the code in t.i._sslverify that OpenSSLCertificateAuthorities somehow gets adapted to IOpenSSLTrustRoot, but it is not really clear to me how this happens.

I know the stock agent doesn't do any cert-checking. I am working with a fork of treq and am experimenting with adding an option to provide a custom Agent.

Any help with the trustRoot argument would be appreciated. If I am going about this the hard way, please let me know that, too.

2 Answers 2

6

Your question here highlights a terrible oversight in the documentation; both in the API documentation, and in the narrative documentation for. If Jean-Paul can't figure out the "right way" to do this, then there is clearly no hope for a regular user. I have filed a bug to correct this oversight.

In the meanwhile, please avoid Jean-Paul's solution. While it is functional, it involves techniques which will almost certainly break without warning in future releases (as he clearly notes). Luckily there are supported ways to do this. If you have a single alternate trust root, Certificate is usable as a value to the trustRoot parameter. You can use it like so (I have tested the following example with Twisted 14.0.2):

from __future__ import print_function
from twisted.web.client import Agent, BrowserLikePolicyForHTTPS
from twisted.internet.task import react
from twisted.internet.ssl import Certificate
from twisted.internet.protocol import Protocol
from twisted.python.filepath import FilePath
from twisted.internet.defer import inlineCallbacks, Deferred

@inlineCallbacks
def main(reactor):
    customPolicy = BrowserLikePolicyForHTTPS(
        Certificate.loadPEM(FilePath("your-trust-root.pem").getContent())
    )
    agent = Agent(reactor, customPolicy)
    response = yield agent.request(
        "GET", "https://your-web-site.example.com/"
    )
    done = Deferred()
    class CaptureString(Protocol):
        def dataReceived(self, data):
            print("Received:", data)
        def connectionLost(self, reason):
            done.callback(None)
    response.deliverBody(CaptureString())
    yield done

react(main)
4
  • 2
    It's a little more complicated to do this if you have multiple alternate acceptable trust roots, and I've filed a Twisted ticket to address that as well: twistedmatrix.com/trac/ticket/7671.
    – Glyph
    Commented Oct 6, 2014 at 23:15
  • Will #7671 also address combining your own trust roots with the system trust roots? There are situations where I'd like to use the OS CA bundle plus my own CA cert. I'm not sure the proposed solution in the ticket accounts for that, unless you can just give the path to wherever the OS bundle is stored.
    – Carl
    Commented Oct 7, 2014 at 17:33
  • I guess it should. Please comment on the ticket for further discussion, though :). (Also, if you wouldn't mind changing this answer to "accepted" - I'd rather people take this advice than Jean-Paul's in this case.)
    – Glyph
    Commented Oct 8, 2014 at 5:00
  • Some discussion of related issues continues here: twistedmatrix.com/pipermail/twisted-web/2014-October/…
    – Glyph
    Commented Oct 8, 2014 at 18:38
-1

IOpenSSLTrustRoot is a bit of a nonsense API.

It is not a public interface itself - so you can't implement your own. If it were, it has no public methods so it's not clear how you might customize its behavior anyway.

Considering the security-sensitive nature of this interface, I bet Twisted will go to great lengths to make sure whatever code you write continues to work - even though the interface is private and there are only private methods on the interface.

If you write some code that depends on this interface always working like it works now then you can probably at least post to the Twisted mailing list and say so and the folks there will probably try not to break your application.

That said, as I pointed out above, every part of this is private. Twisted's stated policy is there are no guarantees of backwards compatibility here. So proceed at your own risk.

In any case, you can write something like this:

from zope.interface import implementer

from characteristic import attributes

from twisted.internet._sslverify import IOpenSSLTrustRoot

@implementer(IOpenSSLTrustRoot)
@attributes(["root_certificate_path"])
class MyCATrustRoot(object):
    def _addCACertsToContext(self, context):
        context.load_verify_locations(self.root_certificate_path)

Then use a MyCATrustRoot instance as the trustRoot argument to BrowserLikePolicyForHTTPS. Make sure you use at least Twisted 14.0.2 because earlier versions actually ignored the trustRoot you passed to BrowserLikePolicyForHTTPS.

For example, if your trusted "CA" certificate is at /foo/ca.pem:

from twisted.web.client import BrowserLikePolicyForHTTPS, Agent
from twisted.internet import reactor

agent = Agent(reactor, BrowserLikePolicyForHTTPS(
    MyCATrustRoot(root_certificate_path="/foo/ca.pem")))
4
  • In your final example, is the MyCATrustRoot class supposed to be used explicitly, or is there some kind of adaption magic going on?
    – Carl
    Commented Oct 2, 2014 at 20:05
  • Also, one more edit to be clear on the nature of private interfaces in Twisted. Commented Oct 2, 2014 at 22:03
  • I had to provide the path using the keywork arg root_certificate_path=, and Agent's first parameter is the reactor, but other than that, it seems to work. I'm accepting this as the answer.
    – Carl
    Commented Oct 3, 2014 at 13:09
  • Thanks. I'll change the code to fix those two bugs. Commented Oct 3, 2014 at 13:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.