The Web DAT fetches pages of data from a web connection. The data should be ASCII-readable. The Web DAT will automatically uncompress any gzip compressed page transfers. XML content is formatted into a readable indented structure, versus one long line normally sent by the server. An Info DAT can be used to obtain properties of the last page retrieved.
There are two main methods of retrieving a page from a web site using the Web DAT:
- Submit and Fetch
The Fetch method simply fetches the page from the internet using the simple protocol "GET", while the Submit and Fetch method can be used for submitting form data to a server. By default the latter method uses the "POST" protocol.
Both methods allow a DAT table input to specify options while fetching. This table should consist of rows of name/value pairs. The first column consists of the names, while the second column consists of the values. The Fetch method simply concatenates the pairs into the specified URL, while the Update and Fetch method posts the pairs to a webserver, before fetching the resulting page.
For example, assume a table with the following contents is connected to the Web DAT:
Parameters - Fetch Page
url - The url address of the web page to be retrieved.
fetch - The data will be fetched when this button is pressed. Use this method to retrieve simple single pages from the internet. By default the Web DAT will stall the process until the whole page has been transferred, or an error occurs. You can do asynchronous downloads using the Asynchronous Fetch option.
method - ⊞ - Currently only POST is implemented, though this will be expanded with other techniques such as GET.
Submit and Fetch
submitfetch - Post all the name/value pairs from the input DAT to the server, then fetch the page specified in the URL parameter.
Use this method to post data to a web server before retrieving the page. The data to be sent is in the form of name and value pairs. It can be specified with a table formatted DAT connected to the first input, where the first column represents data names, and the second column represents data values. The Web DAT will pause until the whole page has been transferred, or an error occurs.
Include Header in Output
includeheader - Includes the HTTP header in the output.
timeout - If this value is 0 the fetch request will never timeout. Any other value is how many milliseconds before the fetch times out.
disconnect - Closes the session.
asyncfetch - Turn on this option to allow the download to occur in the background. You can use a DAT Execute DAT to do something when the data finally arrives.
Verify Peer Certificate
Parameters - Common Page
language - ⊞ - Select how the DAT decides which script language to operate on.
input- The DAT uses the inputs script language.
node- The DAT uses it's own script language.
extension - ⊞ - Select the file extension this DAT should expose to external editors.
dat- various common file extensions.
- From Language
language- pick extension from DATs script language.
- Custom Extension
custom- Specify a custom extension.
customext - Specifiy the custom extension.
wordwrap - ⊞ - Enable Word Wrap for Node Display.
input- The DAT uses the inputs setting.
on- Turn on Word Wrap.
off- Turn off Word Wrap.
|• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •|