Bash script to download set of files based on the list. You will find this a very useful script as you try to use this in many other programs. I will provide some examples to use this script to download images from a site based on keywords. This below script requires single argument which includes list of the HTML links to be downloaded.
#!/bin/bash # Loop through each link while [ `find "$1" -size +0` ] do url=`head -n1 "$1"` head -n1 "$1" > temp wget -Fi temp ERRORCODE=$? # if no error downloading the link if [ $ERRORCODE -eq 0 ]; then sed -si 1d "$1" # if error downloading the link try later # remove from the top and put it in the bottom else echo "ERROR: could not get url" 1>&2 echo "$url" >> "$1" sed -si 1d "$1" fi done