Questions tagged [twisted]
Twisted is an event-driven networking engine, written in Python and implementing many different protocols.
3,427
questions
0
votes
0
answers
13
views
How to implement queries on a page
I'm making a page in python that i want to be able to use queries to go to content. however i cannot find anything about making python code have query support, i only found how request a url with ...
0
votes
0
answers
13
views
how do i fix "ERR_SSL_PROTOCOL_ERROR"?
i made a server with twisted python like below
options = ssl.CertificateOptions(privateKey=cert.privateKey.original,
certificate=cert.original,
verify=False,
...
0
votes
0
answers
12
views
How to deactivate unwanted Twisted log output when using Scrapyd?
When using print method I am receiving log output I haven't seen before.
I guess it's coming from Twisted module which seems to be a part of Scrapyd. I am not using this kind of logging elsewhere and ...
0
votes
0
answers
22
views
i get 'SSL routines', 'ssl3_get_record', 'wrong version number' Error
when i send post request to daemon server using curl it shows below.
SSL3 alert write:warning:close notify
so i tried curl with -k option and then it worked.
i think i need to disable certificate ...
-1
votes
0
answers
19
views
Having problems using dpkg on drozer in Ubuntu (python-twisted dependency error)
I've downloaded drozer from their website and have tried running dpkg on the deb file. However, I've run into some dependency issues. I tried using aptitude to download all the dependencies the dpkg ...
0
votes
0
answers
10
views
Python chat server with selectable specific user to chat
I am trying to write simple chat based desktop support app. The gui is not problem, but I have problem with sending messages. The server sends it to all connected clients, but I want request and ...
-1
votes
0
answers
16
views
how do i decrypt ssl applied data in twisted python?
i found this.
wireshark can decrypt ssl encrypted data.
and there might be a simmillar function in twisted python at dataReceived function.
how do i decrypt ssl applied data?
0
votes
1
answer
91
views
How stream a response from a Twisted server?
Issue
My problem is that I can't write a server that streams the response that my application sends back.
The response are not retrieved chunk by chunk, but from a single block when the iterator has ...
0
votes
0
answers
38
views
Is there a way to stop @repeat_every decorator in FastAPI
I need to create a continuous loop to one of the class Attribute in FastAPI.
For eg.,
class Exampl:
def __init__(self, id_, connect_url='', status='Down'):
"""...
0
votes
1
answer
43
views
"error.ReactorNotRestartable() twisted.internet.error.ReactorNotRestartable"- Binance websocket api
I am getting an issue with websocket. Any example I run from https://github.com/Binance-docs/binance-futures-connector-python/tree/main/examples/websocket/futures gives me the following error - "...
0
votes
1
answer
16
views
how do i send post without verification using Certificate.loadPem?
i'm trying to send post request to daemon which is SSL applied. i want to pass the SSL verification and encrypt with SSL at the same time and here is what i found :
verify=false
but i can't find ...
-1
votes
1
answer
40
views
Approach to Migrate from twisted to fastapi Framework [closed]
I had a requirement of migrating twisted to FastApi.
The old code was completely written in Python 2.7 and used JsonPrc along with twisted client.
Like - [https://stackoverflow.com/a/4738563][1]
Even ...
0
votes
1
answer
49
views
how to run arcade with twisted?
I wish to use the arcade library with the twisted library. but they both have a blocking main loop run() function. what should I do?
I tried using threads so the main loops would run simultaneously. ...
0
votes
3
answers
330
views
Scrapy - ReactorAlreadyInstalledError when using TwistedScheduler
I have the following Python code to start APScheduler/TwistedScheduler cronjob to start the spider.
Using one spider was not a problem and worked great. However using two spiders result into the error:...
0
votes
1
answer
25
views
How to send and receive data on the same line in twisted python?
I am simulating an SFTP server using twisted python.
i want to send the "ftp>" before getting an input from the user on the same line.
i have tried
basic.LineOnlyReceiver.sendLine(self, ...
0
votes
1
answer
75
views
scrapyd deployed in KubeSphere,and when running scrapy selenium got exception:'twisted.internet.error.ReactorAlreadyInstalledError'
I deploy scrapyd in KubeSphere, I got expectation when I run scrapy and selenium:
2022-03-16T12:57:15+0000 [Launcher,1832/stderr] return Crawler(spidercls, self.settings, init_reactor=True)
...
0
votes
0
answers
499
views
running scrapy got the exception :twisted.internet.error.ReactorAlreadyInstalledError: reactor already installed
2022-03-17T08:08:58+0000 [-] Loading /usr/local/lib/python3.9/site-packages/scrapyd/txapp.py...
2022-03-17T08:08:58+0000 [-] Basic authentication disabled as either username or password is unset
2022-...
0
votes
0
answers
25
views
Scrapy and crochet
I know this question has already been discussed, however, I could not find a working solution yet.
I am working on a Django + Celery + Scrapy project, where tasks to scrape are issued by users.
...
0
votes
0
answers
17
views
Sending mail to CC, BCC from python twisted package
I am trying to send mail using twisted sendmail with multiple emails in to_addr. Is there any provision of adding bcc in twisted sendmail functionality?
I am using below mention code
yield sendmail(
...
0
votes
0
answers
236
views
SSL Certification Validation Failed in Binance Connector for Python
I am trying to build an app to detect pumps and dumps in crypto market, so I decided to use binance because of popularity.
So I installed binance-connector by pip3 install binance-connector on macOS ...
0
votes
0
answers
32
views
Is there a way of running two spiders with a Crawl.runner / process and save the results in two separate files?
I have two scrapy spiders in two different scripts
Spiders
Spider1.py
Spider2.py
An example of the code in the spiders is as follows:
from scrapy.crawler import CrawlerRunner
from twisted.internet ...
-1
votes
1
answer
43
views
Cannot stop twisted reactor when calling reactor.stop()
I'm using twisted, PySide2 and qt5reactor to make a simple program that sends data through network. I want the reactor to stop when I closed the PySide2 window:
class MyWindow(QWidget):
--snip--
...
1
vote
1
answer
47
views
How to run two python event loops and share the same memory?
I am using twisted and PySide2 to create an app that can send information through network. However, twisted's reactor and PySide2's QApplication both need an event loop that runs forever.
What I need ...
0
votes
0
answers
33
views
set time for check connection tcp client PYTHON:TWISTED
Im creating an application to receive data from multiple iot devices
Each iot device has a socket server
so i need to create multi socket client
Im using Twisted library for create multiple socket ...
0
votes
1
answer
29
views
How to stop a Twisted Reactor with a GUI event from Eel?
I'm working on building a Python GUI app (using Eel) that uses a Twisted Reactor to continuously make calls to a network connected RFID reader. The data that's collected needs to be parsed and the GUI ...
1
vote
1
answer
123
views
How to run Scrapy spiders in Celery
There are several posts regarding how to setup Scrapy within a Celery task avoiding to restart the Twister reactor to prevent the twisted.internet.error.ReactorNotRestartable error. I have tried ...
0
votes
1
answer
33
views
How to bind Heroku port to scrapyd
I created a simple python app on Heroku to launch scrapyd. The scrapyd service starts, but it launches on port 6800. Heroku requires you to bind it the $PORT variable, and I was able to run the heroku ...
1
vote
0
answers
17
views
Plugin doesn't show up when running twistd
I created a test twistd plugin following the MyProject example here:
https://twistedmatrix.com/documents/current/core/howto/tap.html
but when I run twistd --help from the project root, the myproject ...
0
votes
0
answers
19
views
Unbale to configure jasminSMS gateway User Creation in python
I am trying to create user in jasminSMS as given in documentation.
But in my case I want to create class like UserManagement to manage user Functions
I can make users but unable start / stop the ...
0
votes
0
answers
48
views
Does the scrapy downloader reuse tcp connection?
I have been been stuck on trying to execute an xhr request with scrapy for a lot of days now (do not feel like counting how many - because it is way too many). All my efforts end with a 400 response.
...
0
votes
0
answers
24
views
Python: Scrapy sequential Spider not loading data correctly on start
I am currently having an issue with my scraping project.
I have two spiders, one that gathers URLs and saves them to a CSV, then another that loads up the CSV file to scrape those URLs.
I also have ...
0
votes
1
answer
31
views
How to pass options from PyTest to Twisted.trial?
I have some twisted.trial tests in my project, that is tests inheriting from twisted.trial.unittest.TestCase.
I need to pass some trial options to my test, specifically it is --reactor option of ...
0
votes
1
answer
38
views
Python Twisted: SSL Example: How to avoid SSL passphrase?
I'm following a Python twisted tutorial in which an echo server is being set up in order to talk to a client over a ssl transport layer. The end goal is to obtain an encrypted connection towards our ...
0
votes
0
answers
39
views
Delay a Deferred's run in Python Twisted
I'm using an API which limits the amount of requests I make in a second. Therefore, I tried creating a "Pacer" module, that will delay Deferred's which request data from the API when needed.
...
0
votes
0
answers
10
views
Twisted services in an existing reactor
I'm trying to combine two existing codebases which use twisted in different ways, and I'm not quite sure how to begin.
In the one case, I have a kivy application integrated with a twisted reactor more ...
0
votes
1
answer
145
views
Scrapy Splash crawler ReactorNotRestartable
I've developed a Scrapy Splash scraper using Visual Studio Code on Windows 10.
When I run my scraper like so without a runner.py file it works and generates the scraped content int "out.json"...
-1
votes
1
answer
60
views
Twisted generating new clients for reactor
Generally in Twisted (Python), you define some listeners or connections or looping operations and add them to the reactor and then call reactor.run(). Is there any way to add new connections from ...
-1
votes
1
answer
77
views
Unit Test for twisted addCallback with nested method
I have a situation to write a unit test for the below code, please help me out here.
def method1(self):
self.method2()
def method2(self):
def method2_success(result):
if ...
-1
votes
1
answer
253
views
Running Scrapy with Flask - Unhandled error in EventualResult
I have a scrapy project that works fine and returns resutls i want.
The spider code is:
class ExampleSpider(scrapy.Spider):
name = 'example'
myBaseUrl = ''
start_urls = []
def __init__(...
-1
votes
1
answer
31
views
Ensuring SQL query order in multithread pool
I am making a small game and using MySQL as the database. I am having a slight issue with saving because of a multithreaded pool. When I submit inserts/deletes, say for adding or deleting items, there'...
1
vote
0
answers
28
views
getting errror in library itself twisted 21.7.0
Sep 27 09:25:00 ip-172-31-36-82 python3[28020]: Traceback (most recent call last):
Sep 27 09:25:00 ip-172-31-36-82 python3[28020]: File "/home/ubuntu/.local/lib/python3.8/site-...
0
votes
0
answers
30
views
What effect does unclean (forced) shutdown have on a spider's job?
In the middle of huge website's API scraping Job (~1.5 million endpoints), it shutdown -the unclean way (double press ctrl-c).
>>> scrapy crawl somespider -s JOBDIR=crawl
loads of requests...
...
0
votes
1
answer
342
views
How to run async Twisted reactor and asyncio loops infinitely
I have two libraries, both of which provide listener functions (which run on different ports) and listen for network activity. (My program provides a interface layer between them.)
The issue is that ...
0
votes
0
answers
54
views
Using Python Twisted, write a DNS server that is able to respond to incoming APL queries
Please help with implementing a dns server using python twisted that is capable of handling APL request.
i have tried working with https://twistedmatrix.com/documents/16.5.0/names/howto/custom-server....
0
votes
1
answer
47
views
How to communicate with a python Twisted server from Java successfully?
These questions may sound silly, but I am new to this networking thing.
I have been trying for quite a few days now to implement a client that works with a Twisted server, but I am failing to get any ...
3
votes
1
answer
122
views
scrapy : parallel AND sequential running of spider
I have a scrapy project with multiple spiders. Some take minutes, some take hours, and anything in between - however that elapsed time is usually about the same for each run - so you can assume that ...
0
votes
0
answers
20
views
How to restart twisted.trial._dist.disttria.DistTrialRunner?
I try to use twisted.trial._dist.disttrial.DistTrialRunner as a runner, after some cases fail I want to re-run the fail cases, but it's not restart and get an error:
from twisted.trial._dist.disttrial ...
0
votes
1
answer
53
views
Get last row id from runQuery
I am inserting with twisted runQuery into mysql. How to i get the last inserted row id from the transaction? Actually i do another sql query after insert, but this isnt really elegant.
according to (...
0
votes
1
answer
54
views
Get results of Scrapy spiders in variable
I try to run Scrapy spider and some SDK call to another resource inside Django. The main idea collect results from both of them in one list once it will be ready and output it to view. SDK is working ...
0
votes
0
answers
39
views
Twisted application using TxMongo have Mongo DB Atlas connections spiking to maximum limit (1500)
I have a USSD menu app, built using Twisted and txmongo (used for connecting to Mongo DB Atlas).
I have the following method:
@inlineCallbacks
def get_mongodb_connection_pool():
db = yield ...