Layer 4-7: Web Browsing with Python

Testing Prerequisites

You must be running at least LANforge version 5.5.1 on Linux and/or Windows resources.

To ensure we can get stats from Python, you should make sure the resources in question have access to the internet. Keep in mind, various websites will respond to a Python web scraping bot differently.

Linux

Open a terminal run the following command as user lanforge:
pip install selenium
or for Debian-based systems (like the ct521-at7): sudo apt install chromium chromium-driver pythyon3-selenium

Windows

You will need Chrome and a Chromedriver for the Chrome version installed. These should have been installed with windows_lf_setup.ps1. Please open a terminal in Windows and nagivate to the Downloads directory.

Navigate to this directory in PowerShell with the following command:
cd C:\Users\Administrator\Downloads

List files with this command:
ls

Inside this current directory, you should see files like: chrome.msi and chrome_driver.zip. To test if these are installed correctly, run this command in the PowerShell terminal:
Test-Path "C:\Program Files (x86)\LANforge-Server\chromedriver-win64" -PathType Container

This should return the value True. If not, please re-run the windows_lf_setup.ps1 script in PowerShell, or contact support for help.

Creating a Web Browsing Endpoint:

1. Navigate to the Layer 4-7 tab in the LANforge.
2. Click the Create button.
3. A new window should pop-up.
   1. [a] Provide a Name for the Endpoint.
   2. [b] Select the LANforge Resource to run the automation on.
   3. [c] Select the Port drop down for the resource in step 2b.
   4. [d] Select URLs per 10m, Max Speed, URL Timeout, Quiesce, Quiesce after URLs, and Quiesce After Time values.
   5. [e] Enter URL, or a text file containing URL's. To use a text file of URL's, you must manually copy the file to the machine's (the resource's) /home/lanforge directory.
      See also: Layer 4 user guide
   6. [f] For Source/Dest File, use /dev/null for linux based machines, and NUL for Windows based machines.
   7. [g] Ensure that Real Browser checkbox is filled.
4. Click the Apply button in the Creation Window.
5. Click Start.

Observing Statistics:

1. Navigate to the Layer 4-7 tab in LANforge.

2. Wait for Layer 4-7 Endpoint to begin.

   Depending on the web site and other parameters selected (like: Quiesce After Time, or Quiesce After URLs), the statistics will take a few seconds to show up.
   

3. To view the statistics on the machine, navigate to the Resource Mgr tab and connect using VNC. See also: Connect to LANforge using VNC.

   Remmina works well for Linux based devices. RealVNC works well for Windows based devices. A Windows Terminal session is shown below.
   

Stopping the test:

1. Once the desired time or URL count (or other metric) has been completed, click Stop Test.

Troubleshooting:

1. If the test fails to run, check the DNS nameserver settings.
   

   Linux

   open a bash terminal and type:
   host -v google.com
   This displays the relevant information such as:
   Received 249 bytes from 75.75.75.75#3 in 44 ms
   (The number of bytes, nameserver and ms will vary.) Please check that the nameserver settings are correct with:
   cat /etc/resolv.conf
   

   Windows

   In PowerShell, use this command:
   nslookup google.com
   This displays as:

   Server: cdns01.comcast.net
   Address: 75.75.75.75
   Non-authoritative answer:
   Name:    google.com

   Please check DNS settings on Windows with the following command:
   ipconfig /all
   Find your DNS namserver on the line starting with DNS Servers . . . . . . . . . . . :

2. If the test continues to fail to run, check default routing.
   

   Linux

   Open a bash terminal and type:
   mtr -r -c 5 google.com
   Errors will look like this: mtr: connect: Network is unreachable.
   

   Windows

   Open a PowerShell terminal and type the following command:
   tracert 8.8.8.8
   If tracert returns nothing, or fails immediately then a default route may be missing.

3. Should other issues appear, please contact support for help.