Perl Tutorials - Herong's Tutorial Examples - v6.01, by Dr. Herong Yang
LWP-UserAgent-GET.pl - Sending a GET Request
This section provides a tutorial example on how to send a GET request to a Web server and dump both request and response messages.
We have learned 3 classes, LWP::UserAgent, HTTP::Request, and HTTP::Response in previous tutorials, with some simple tests. Now let's try to write more real Perl program to send a GET request to any given URI and dump both request and response.
Here is my first version of LWP-UserAgent-GET.pl:
#- LWP-UserAgent-GET.pl #- Copyright (c) HerongYang.com. All Rights Reserved. use LWP::UserAgent; my ($uri) = @ARGV; $uri = 'http://herongyang.com/' unless $uri; my $browser = new LWP::UserAgent(); my $response = $browser->get($uri); my $request = $response->request(); print($request->as_string()); print($response->as_string()); exit;
Test 1: When testing the program on google.com, I got the following:
herong> LWP-UserAgent-GET.pl http://www.google.com GET http://www.google.com User-Agent: libwww-perl/6.05 HTTP/1.1 200 OK Cache-Control: private, max-age=0 Connection: close Server: gws Content-Type: text/html; charset=ISO-8859-1 Expires: -1 Alternate-Protocol: 80:quic Client-Response-Num: 1 P3P: CP="This is not a P3P policy! See http://www.google.com/suppo... Set-Cookie: PREF=ID=fb56a6bf2677de5a:FF=0:TM=1393607257:LM=1393607... Set-Cookie: NID=67=iolNVOo3ULZAHWxwv3P-LQhF5GRZvNQhgZyEedVVHOzGQzV... Title: Google X-Frame-Options: SAMEORIGIN X-Meta-Description: Search the world's information, including webp... X-Meta-Robots: noodp X-XSS-Protection: 1; mode=block <!doctype html><html itemscope="" itemtype="http://schema.org/WebP... ...
LWP-UserAgent-GET.pl worked as expected. Both request and response are printed out.
Test 2: When testing the program on facebook.com, I got the following:
herong> LWP-UserAgent-GET.pl http://www.facebook.com GET https://www.facebook.com/unsupportedbrowser User-Agent: libwww-perl/6.05 HTTP/1.1 200 OK Cache-Control: private, no-cache, no-store, must-revalidate Connection: close Content-Type: text/html; charset=utf-8 Client-Response-Num: 1 Client-SSL-Cert-Issuer: /C=US/O=VeriSign, Inc./OU=VeriSign Trust N... Client-SSL-Cert-Subject: /C=US/ST=California/L=Palo Alto/O=Faceboo... Client-SSL-Cipher: ECDHE-RSA-AES128-GCM-SHA256 Client-SSL-Socket-Class: IO::Socket::SSL Client-Transfer-Encoding: chunked Link: <https://www.facebook.com/unsupportedbrowser>; media="handhe... Link: <https://fbstatic-a.akamaihd.net/rsrc.php/yV/r/hzMapiNYYpW.i... P3P: CP="Facebook does not have a P3P policy. Learn why here: http... Refresh: 0; URL=/unsupportedbrowser?_fb_noscript=1 Set-Cookie: reg_fb_ref=https%3A%2F%2Fwww.facebook.com%2Funsupporte... Title: Update Your Browser | Facebook X-Content-Type-Options: nosniff X-FB-Debug: zKaAOvfvJwkkb2nP7VcGQwahBGkRIZ1frxAdPRL6Ngs= X-Frame-Options: DENY X-Meta-Charset: utf-8 X-Meta-Referrer: default X-Meta-Robots: noodp, noydir X-XSS-Protection: 0 ... <!DOCTYPE html> <html lang="en" id="facebook" class="no_js"> <head><meta charset="utf-8" /><script>function envFlush(a){functio...
It worked too. But there were 2 surprises in the output:
With this information, I can improve my LWP-UserAgent-GET.pl program to make facebook.com happy now, see the next tutorial.
Table of Contents
Data Types: Values and Variables
Expressions, Operations and Simple Statements
Name Spaces and Perl Module Files
Hard References - Addresses of Memory Objects
Objects (or References) and Classes (or Packages)
Typeglob and Importing Identifiers from Other Packages
String Built-in Functions and Performance
File Handles and Data Input/Output
Open Directories and Read File Names
File System Functions and Operations
Socket Communication Over the Internet
XML::Simple Module - XML Parser and Generator
SOAP::Lite - SOAP Server-Client Communication Module
Perl Programs as IIS Server CGI Scripts
CGI (Common Gateway Interface)
XML-RPC - Remote Procedure Call with XML and HTTP
RPC::XML - Perl Implementation of XML-RPC
Integrating Perl with Apache Web Server
CGI.pm Module for Building Web Pages
►LWP::UserAgent and Web Site Testing
►LWP-UserAgent-GET.pl - Sending a GET Request
LWP-UserAgent-GET-Redirect.pl - Following HTTP Redirects
http-equiv="Refresh" Meta Tag not Followed
LWP-UserAgent-POST.pl - Posting Form Data
post() Method not Following Redirect Location
LWP-UserAgent-POST-Redirect.pl - Posting with Redirects
LWP-UserAgent-Request.pl - GET, POST and Cookies
LWP-UserAgent-Request.pl - Login to facebook.com
HTTP::Cookies save() not Saving Temporary Cookies
LWP-UserAgent-Request-Cookies.pl - Sending Request with Cookies
Converting Perl Script to Executable Binary