[solved] Logging into HTS with python

[salebete]

EDIT: SEE MY LAST POST TO SEE A CORRECTED VERSION OF THE PYTHON CODE

Hi,
I'm trying to solve the first programming exercise, but I'm stuck with the authentification part of the problem : getting the program to login with HTS.
For that I followed the instructions of the httplib2's tutorial

Here is a trimmed version of my script (without the algorithmic part solving the problem)
I tried different variations without success. I would be very happy if someone tells me what I'm missing.

Code: Select all

#!/usr/bin/python
# -*- coding: iso-8859-15 -*-

import httplib2
import urllib
import urllib2
   
def main():
    http = httplib2.Http()
   
    url = "http://www.hackthissite.org/user/login"
    body = {'username': '*******', 'password': '*******'}
    headers = {'Content-type': 'application/x-www-form-urlencoded'}
    response, content = http.request(url, 'POST', headers=headers, body=urllib.urlencode(body))
    headers['Cookie'] = response['set-cookie']
   
    url = "http://www.hackthissite.org/missions/prog/1/"
    response, content = http.request(url, 'GET', headers=headers)

    if content.find("Login Required") > 0:
        print "Login failed"
    else:
        print "Login succeeded"

if __name__=='__main__':
    main()

[mischief]

does python have cURL? maybe using cURL would be easier.. :p

[Zer0--10xin]

perl is quicker

[Defience]

It isn't necessary to do this. What you can do though is log on to hts, get your cookie and add that to your code, then run it as if you are already logged in using: import urllib,urllib2,re

[salebete]

I finally figured out what was the problem. I looked at the POST transaction using Tamper Data and tried to mimic as closely as possible with the program. Finally I came to the conclusion that the 'Referer' information in the POST header is very important. Without that information the server returns a 404 not found page !

I also made a version with pycurl. The code is quite ugly, but, as you know, it would have been even worse with PERL (albeit quicker).

Version with httplib2

Code: Select all

#!/usr/bin/python
# -*- coding: iso-8859-15 -*-

import httplib2
import urllib
import urllib2
   
def main():
    http = httplib2.Http()
   
    url = "http://www.hackthissite.org/user/login"
    body = {'username': '********', 'password': '********'}
    headers =  {
            'Host': 'www.hackthissite.org',
            'User-Agent':'Mozilla/4.0 (compatible; MSIE 6.0)',
            'Referer':'http://www.hackthissite.org/user/login',
            'Content-Type':'application/x-www-form-urlencoded',
            'Accept-Encoding':'gzip,deflate',
        }
   
   
    response, content = http.request(url, 'POST', headers=headers, body=urllib.urlencode(body))
    fp = open('data.html', 'w')
    fp.write(content)
    fp.close()
    headers['Cookie'] = response['set-cookie']
   
    url = "http://www.hackthissite.org/missions/prog/1/"
    response, content = http.request(url, 'GET', headers=headers)

    print content


if __name__=='__main__':
    main()


Version with pycurl

Code: Select all

#!/usr/bin/python
# -*- coding: iso-8859-15 -*-

import pycurl
import StringIO

def main():
    buffer = StringIO.StringIO()
   
    headers =  [
            'Host: www.hackthissite.org',
            'User-Agent: Mozilla/4.0 (compatible; MSIE 6.0)',
            'Referer: http://www.hackthissite.org/user/login',
            'Content-Type: application/x-www-form-urlencoded',
            'Accept-Encoding: gzip,deflate',
        ]
    url = 'http://www.hackthissite.org/user/login'
    curl = pycurl.Curl()
    curl.setopt(pycurl.URL, url)
    curl.setopt(pycurl.COOKIEJAR, 'cookies.txt')
    curl.setopt(pycurl.COOKIEFILE, 'cookies.txt')
    curl.setopt(pycurl.POSTFIELDS, "username=********&password=********")
    #curl.setopt(pycurl.VERBOSE, 1)
    curl.setopt(pycurl.FOLLOWLOCATION, 1)
    curl.setopt(pycurl.POST, 1)
    curl.setopt(pycurl.HTTPHEADER, headers)
    curl.setopt(pycurl.WRITEFUNCTION, buffer.write)
    curl.perform()
       
#   buffer.flush()
    buffer.close()
    buffer = StringIO.StringIO()
    curl.setopt(pycurl.WRITEFUNCTION, buffer.write)
    url = "http://www.hackthissite.org/missions/prog/1/"
    curl.setopt(pycurl.HTTPGET, 1)
    curl.setopt(pycurl.URL, url)
    curl.perform()
    curl.close()
   
    content = buffer.getvalue()
    print content

if __name__=='__main__':
    main()


[Defience]

Well, glad to see you figured it out and learned in the process but again, for the hts programming missions it isn't necessary to go through all of that. Good job.

[turbotax]

Defience,

Noob question to follow up. I'm trying to figure out how you would do it the way you're proposing. I've been doing it like the OP. Some pseudo code please?

[Defience]

Defience,

Noob question to follow up. I'm trying to figure out how you would do it the way you're proposing. I've been doing it like the OP. Some pseudo code please?

Log in to HTS and get your cookie.....

Code: Select all

import urllib2

url = ' website.here'
strSession = ' cookiehere'
dicHeaders = {'COOKIE': strSession}
req = urllib2.Request(url, None, dicHeaders)


Next, still using urllib2, open req, read it, and print out the context of the page. Let me know if you need more info on it.

[turbotax]

Nope, that's perfect. Thank you so much.

[SiWi]

Hallo, I got a problem with this stuff as well(want to use it for mission 12).
The code I'm currently using is the following:

Code: Select all

import urllib2
import urllib
def connect(subpage):
    url="http://www.hackthissite.org/"+subpage
    url+="index.php"
    data = urllib.urlencode(dict(solution='Hallo'))
    req = urllib2.Request(url,data, {'REFERER': 'http://www.hackthissite.org/missions/prog/12/index.php','COOKIE':"PHPSESSID=xxxxxx; path=/; domain=www.hackthissite.org"})
    response = urllib2.urlopen(req)
    print response.read()
connect("missions/prog/12/")


The problem is that I don't get the "your answer is false/wrong" page, but the normal mission page.
What do I have to change?