Quantcast

Jump to content


Photo

Mechanize problem?

mechanize python programming url

  • Please log in to reply
5 replies to this topic

#1 RitzWin

RitzWin
  • 241 posts

Posted 10 June 2012 - 11:58 AM

I'm getting this error with my daily doer:

URLError: <urlopen error [Errno 10054] An existing connection was forcibly closed by the remote host>

I was just about to release the newest version but I've come across this error randomly when arriving at a Browser.submit() function using mechanize. I'm not sure what's going on here and google's not being entirely helpful about it :(

Of course I could catch it and try it again in a few seconds... but I'm not sure that's the best way to go about things...

Even if you have no python experience I'd appreciate some feedback if you've seen this error before. It seems like maybe I'm getting logged out before I'm submitting a form? I'm really not sure any help is appreciated!

I'm also pretty sure that I've not seen this error when I run it in linux- which is odd, it might be the version of mechanize?

#2 Hydrogen

Hydrogen
  • Neocodex Co-Founder

  • 22213 posts


Users Awards

Posted 10 June 2012 - 12:02 PM

The Neopets web server is closing the connection to your client program. Depending on how you are accessing their site, which HTTP headers you are sending to the server, etc, you might even be requesting the Neopets server to do this. Can you print the HTTP headers that you are sending? Perhaps you can get them through tcpdump if not through the program itself?

#3 Melchoire

Melchoire
  • 5284 posts


Users Awards

Posted 10 June 2012 - 10:36 PM

Take a look at the HTTP requests you're making to neopets, using wireshark or like hydro said just dump the mechanize request to a text-file. Then compare it to the requests you send if you use firefox. I guess to remain stealth you wanna mimic a regular web-browser as closely as possible.

Do you guys spoof the user-agent field to match firefox when using mechanize?

#4 RitzWin

RitzWin
  • 241 posts

Posted 11 June 2012 - 07:59 AM

Take a look at the HTTP requests you're making to neopets, using wireshark or like hydro said just dump the mechanize request to a text-file. Then compare it to the requests you send if you use firefox. I guess to remain stealth you wanna mimic a regular web-browser as closely as possible.

Do you guys spoof the user-agent field to match firefox when using mechanize?


Yes, to spoofin the user-agent.

I'm not sure if the current version of my daily-doer has this problem... I'll have to compare both requests (I don't think I changed anything in this area though).

I'll get back in a few days when I have time to check this.

#5 RitzWin

RitzWin
  • 241 posts

Posted 13 June 2012 - 11:21 AM

I've been noticing that Neopets hasn't been playing friendly lately for me (don't know if it's just me), but sometimes when I click a form button it takes forever, and then never replies. I believe this is just mechanize hitting the same issue.

To fix it I've caught the error and retry after a few seconds.

Thx everyone.

/edit I'm thinking of just making a wrapper for the submit function, so that it retries if it gets this error- probably the safest/easiest way to deal with it :)

Edited by RitzWin, 13 June 2012 - 11:27 AM.


#6 Melchoire

Melchoire
  • 5284 posts


Users Awards

Posted 13 June 2012 - 04:38 PM

I've been noticing that Neopets hasn't been playing friendly lately for me (don't know if it's just me), but sometimes when I click a form button it takes forever, and then never replies. I believe this is just mechanize hitting the same issue.

To fix it I've caught the error and retry after a few seconds.

Thx everyone.

/edit I'm thinking of just making a wrapper for the submit function, so that it retries if it gets this error- probably the safest/easiest way to deal with it :)


If it's happening frequently you can also adjust the time-out so that it gets caught faster.



Also tagged with one or more of these keywords: mechanize, python, programming, url

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users