How To Submit Forms With PHP
Ever wonder how people can create and login to hundreds of myspace accounts or blogger blogs? It would be naive to assume that they maintain these accounts by hand.
The truth is, when your operations get to a certain point you have to start automating several of the mundane and repetitive tasks.
In order to automate most web-based tasks you need to know how to submit a form programmatically, and I’m going to show you exactly how to do just that.
I’m going to assume that you have PHP installed on your server, as well as libcurl. I’m also going to assume that you have a basic understanding of PHP and HTML forms. If you don’t, there are plenty of good tutorials.
I’d also suggest that you download the LiveHTTPHeaders Firefox extension. This plugin will tell you the exact parameters that your browser is sending when you submit any form. The alternative to installing and using this plugin is digging through the source of each page and finding the form parameters yourself.
To show you how to submit a form using PHP and cURL, I’m going to login to Technorati. I chose Technorati because it uses very few parameters when you login, which will keep our code as clean as possible.
The first thing we have to do is figure out what Technorati expects when we login and this is where the LiveHTTPHeaders plugin comes in handy. After loading the Technorati homepage in your browser, start the LiveHTTPHeaders plugin, and then login using your account. The LiveHTTPHeaders window will show you all the interactions between your browser and technorati.com.
The only information in the LiveHTTPHeaders window that we are concerned with is the first line and the last line of the first “paragraph.” The first line in my window looks like this:
http://technorati.com/login.php
This is the page that is handling the form. The last line of the first paragraph looks like this for me:
username=wagerank&password=%pass%
Note that I changed my real password to %pass%.
Next, we have to prepare our server to pretend it’s a browser, which is easier than it sounds. All you have to do is create an empty file called “cookies.txt” in the directory where you want to run your script.
The next thing we need to do is actually write the PHP script that will submit this form for us. I found myself writing the same cURL code over and over again, so I finally decided to encapsulate it in a function. The function is called hitForm and in the simple case it takes two parameters: the page that is handling the form and the form parameters. Here is the code:
function hitForm($loginURL, $loginFields, $referer=”") {
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, “cookies.txt”);
curl_setopt($ch, CURLOPT_COOKIEFILE, “cookies.txt”);
curl_setopt($ch, CURLOPT_URL, $loginURL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $loginFields);
$ret = curl_exec($ch);
curl_close($ch);
return $ret;
}
I’m not going to explain what every single line does, but if you read a tutorial on cURL you should be able to figure it out.
Ok, now that we have that function defined, we just have to pass it the data we pulled out of the LiveHTTPHeaders window:
hitForm(”http://technorati.com/login.php”, “username=wagerank&password=%pass%”);
Everything you do after that call will be done as if you were sitting in front a browser, logged in to technorati.
To illustrate, I created a small script that will log you in to Technorati, and go through all the steps to add WageRank to your list of favorite blogs.
Just download the script, change the username and password to your details, rename the file to technorati.php, upload it to your server, create an empty ‘cookies.txt’ file in the same directory, and then load the file in your browser (e.g. http://yourserver.com/technorati.php).
Congratulations! You just automated your first task.
Logging into MyBlogLog and joining the WageRank Community is left as an exercise for the reader.
If you run into any problems, just leave a note in the comments below and I’ll try to help out.
Disclaimer: I take no responsibility for your account being banned due to your use of this script. Any code or advice I give away is for instructional purposes only.
Related Posts
Welcome, Earners Blog Readers
Welcome, Net Business Blog Readers
Adding Domains To WHM With PHP
Links From Google Reader
Wordze API PHP Script
[…] It’s not entirely clear what you want to do, but I think you want to submit comments from a program? If so, this should help: How To Submit Forms With PHP __________________ WageRank.com […]
Hi,
Thanks for the curl tutorial.
I’ve got a question, and I’m sure the solution is very simple; I just can’t find the answer. I have the following code (my username changed to MYUSERNAME and my password changed to MYPASSWORD):
and I load it in my browser and I expect to see Technorati loaded, with me being logged in. But the reality is, I only see a blank white page, and I have no idea what’s wrong?
I would really appreciate your reply.
Thanks.
Hi Frank,
Here are a few things to check:
1. Is the browser still loading? Sometimes it takes a little while for the script to fetch the Technorati page and bring it back to you. Give it a few seconds (at most 10) and see if that works.
2. Check to see if you have PHP and cURL installed. To do this, add the following to the script immediately above the $username line:
print hitPage(”http://www.google.com”);
exit();
This will retrieve Google and then exit the script. If that doesn’t work, I would check with your host to see if you have PHP and cURL enabled.
If neither of those help, let me know.
Hi cdc,
Thanks a lot for the reply. Yes I know for sure that I have cURL enabled, and I also made sure that the browser finished loading. What’s even more strange is that, as you know, that little script that I posted did not load anything, but if I run the whole code (http://www.wagerank.com/downloads/technorati.txt) it works fine, and I could even see the technorati page loaded (with my being logged in) with your website being added to my Technorati account, so that definitely means that the login process did work, but somehow I was not able to see it. In other words, if I run your partial script (only logging in), I can’t see myself being logged in. But if I run the full script (i.e: logging in AND adding wagerank.com to my Technorati account), I could see everything. Anyway does it work on your end? if you just leave the function hitForm and the login details intact and do print hitForm($loginURL, $loginFields); does it show anything?
Oh one more thing, I have a question posted at http://www.experts-exchange.com/Web/Web_Languages/PHP/Q_22144526.html?cid=239#a18442332 and if you could give me some pointers, I would really appreciate it. The question is very simple (basically I just want to know how to save the HTML code (to a file) of a webpage that can be retrieved only after logging in to a certain website) and I hope it won’t take much of your time. Thanks.
I checked out that thread and it looks like you solved your problem over there. The reason why nothing was printing if you only logged in, but it would print if you go through the whole script, is because I’m only printing out the last part (when you add WageRank to your favorites). If you add the print command to any of the hitPage lines you’ll see that page printed as well.
Yes, I think my mistake was that I was doing print hitForm instead of print hitPage. Thanks for clarifying though.
Would the techniques that you describe work on just any website, or are some websites just impossible to work with? I tried doing a similar thing: what I did was I tried to post a message from my Yahoo! 360 account to my another Yahoo! 360 account and it didn’t seem to work. The URLs there are also very long with so many parametres, and may be a different technique is required?
It should be the same technique, but you may have to be more legit. Technorati was one of the easiest forms to submit which is why I used it as an example. Blogger and MySpace, for example, take extra measures to keep bots out. I don’t have any experience with Yahoo, but you probably want to try using the referer and user agent functionality of cURL.
Hi — do you offer custom development services? I have a project similar to this post I’m trying to find a developer for. Thanks
I have some doubt can any one clarify. If you are using CURL with postfields will it will be applicable for both windows/lynux based as well as for command prompt. Because I used one when running in browser its working fine and getting the string output but when running in the command prompt I am getting the output as well as the html tags are printed as it is which I dont want in command prompt. Give me a reply.
Thank You
K.Chandran
Hi,
The behavior you are seeing is correct. The HTML is output in both cases; the only difference is that your browser knows how to interpret it.
where is the script????????????
http://www.wagerank.com/downloads/technorati.txt
[…] It’s pretty simple, especially if you already read and understood my post detailing how to submit forms with PHP. […]
[…] about a year ago and found their API cumbersome and unintuitive. If you’ve already learned how to submit forms with PHP, using the Wordze API will be […]
I NEED TO POST COMMENTS TO MYSPACE. I’ve had several developers working on this and they have had a very hard time getting past Myspace’s measures to prevent this–either way, we have the base code, but need to get past whatever myspace has done to prevent such bots. We also have the codes for friend requests, messages and bulletins. We got all those functions to work.
Anyway I will pay anyone who can deliver the auto comment posting function for Myspace. I’m also willing to trade the similar code for messages, bulletins and friend friend requests. Please email me at jamesgillmore@gmail.com if you can do this.
James
thanks for the great tutorial.
I am a fresh developer, and have meagre knowledge of the field.I wish to know however, how do you extend the above technique? How do I embedd a client’s dynamic form(built using ASP.net) on my website, simultaneously updating my database on each fill of the form?
Please find time to answer this soon.
[…] isn’t that difficult. I forget who owns this blog, but there’s a good tutorial/script here: How To Submit Forms With PHP __________________ [FOR SALE] 2,000 Tutorial Site URLs $7 [FOR SALE] 16,000+ Golf Courses + […]
Thank you for the great, well written tutorial. I have a question about form submission, and I’m hoping the answer is very simple.
I’m trying to submit a search to Google, so use the following code:
Google replies with a 501 (Not implemented - the server is unable to process your request). Please can you assist? What am I doing wrong?
Thank you.
Thank you for the great, well written tutorial. I have a question about form submission, and I’m hoping the answer is very simple.
I’m trying to submit a search to Google, so use the following code:
Google replies with a 501 (Not implemented - the server is unable to process your request). Please can you assist? What am I doing wrong?
Thank you.
Well pretty clever way to get people to favorite WageRank!
hi,
your script works perfect, HOWEVER
i noticed that when i go to the actual technoratic site, i am not logged in. how can i make it so that when i run this script, i can come back to it later and i will be logged in already ?
i suspect its because of cookies ?
thank you.
sorry i meant coming back to the technorati site and finding that i am logged in….
right now i use this script and i get logged in.
go to technorati’s actual site
and i am not logged in.
also this script will not work
when say httplightreader returns this
vb_login_username=daasdasdf&vb_login_password=&s=&do=login&vb_login_md5password=7923449a757c6cbe0234c22&vb_login_md5password_utf=7946d19234234e0d29c3a2c22
in such sites, its impossible to use curl to authenticate.