Linux downloader customizable

Bash script to download set of files based on the list. You will find this a very useful script as you try to use this in many other programs. I will provide some examples to use this script to download images from a site based on keywords. This below script requires single argument which includes list of the HTML links to be downloaded.


# Loop through each link
while [ `find "$1" -size +0` ]
url=`head -n1 "$1"`
head -n1 "$1" > temp
wget -Fi temp

# if no error downloading the link
if [ $ERRORCODE -eq 0 ]; then
sed -si 1d "$1"

# if error downloading the link try later
# remove from the top and put it in the bottom
echo "ERROR: could not get url" 1>&2
echo "$url" >> "$1"
sed -si 1d "$1"


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s