Tagged: Bash Toggle Comment Threads | Keyboard Shortcuts

  • jess 7:03 am on May 20, 2015 Permalink | Reply
    Tags: Apache, Bash, Logs   

    Apache log one-liners using tail, awk, sort, etc. 

    Good bunch of samples with other examples found at: https://blog.nexcess.net/2011/01/21/one-liners-for-apache-log-files/

    # top 20 URLs from the last 5000 hits
    tail -5000 ./transfer.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20
    tail -5000 ./transfer.log | awk '{freq[$7]++} END {for (x in freq) {print freq[x], x}}' | sort -rn | head -20
     
    # top 20 URLS excluding POST data from the last 5000 hits
    tail -5000 ./transfer.log | awk -F"[ ?]" '{print $7}' | sort | uniq -c | sort -rn | head -20
    tail -5000 ./transfer.log | awk -F"[ ?]" '{freq[$7]++} END {for (x in freq) {print freq[x], x}}' | sort -rn | head -20
     
    # top 20 IPs from the last 5000 hits
    tail -5000 ./transfer.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -20
    tail -5000 ./transfer.log | awk '{freq[$1]++} END {for (x in freq) {print freq[x], x}}' | sort -rn | head -20
     
    # top 20 URLs requested from a certain ip from the last 5000 hits
    IP=1.2.3.4; tail -5000 ./transfer.log | grep $IP | awk '{print $7}' | sort | uniq -c | sort -rn | head -20
    IP=1.2.3.4; tail -5000 ./transfer.log | awk -v ip=$IP ' $1 ~ ip {freq[$7]++} END {for (x in freq) {print freq[x], x}}' | sort -rn | head -20
     
    # top 20 URLS requested from a certain ip excluding, excluding POST data, from the last 5000 hits
    IP=1.2.3.4; tail -5000 ./transfer.log | fgrep $IP | awk -F "[ ?]" '{print $7}' | sort | uniq -c | sort -rn | head -20
    IP=1.2.3.4; tail -5000 ./transfer.log | awk -F"[ ?]" -v ip=$IP ' $1 ~ ip {freq[$7]++} END {for (x in freq) {print freq[x], x}}' | sort -rn | head -20
     
    # top 20 referrers from the last 5000 hits
    tail -5000 ./transfer.log | awk '{print $11}' | tr -d '"' | sort | uniq -c | sort -rn | head -20
    tail -5000 ./transfer.log | awk '{freq[$11]++} END {for (x in freq) {print freq[x], x}}' | tr -d '"' | sort -rn | head -20
     
    # top 20 user agents from the last 5000 hits
    tail -5000 ./transfer.log | cut -d\  -f12- | sort | uniq -c | sort -rn | head -20
     
    # sum of data (in MB) transferred in the last 5000 hits
    tail -5000 ./transfer.log | awk '{sum+=$10} END {print sum/1048576}'
    
     
  • jess 7:56 pm on June 11, 2010 Permalink | Reply
    Tags: Bash, ,   

    Mac OSX 10.6 Shell Script to Configure, Launch, and Shutdown Screen Sharing 

    Apple’s MacOS has a tidy built in remote access client and service generically known as “Screen Sharing”. Yay! This works great for headless machines, testing servers, and annoying your spouse. The paranoid and performance conscious user may not want these services continuously running. Rogue services hanging out, listening, and waiting for some punk to probe them in an Internet dark alley. This cobbled together shell script might be for you!

    This shell script has four simple commands (I use sudo because I really should):

    • # sudo ./share-screen.sh start
    • # sudo ./share-screen.sh stop
    • # sudo ./share-screen.sh allow {username}
    • # sudo ./share-screen.sh deny {username}

    I use this in terminal. Make sure you put a copy of this script on the target machine and make it executable. First I make sure that Remote Login is active on the target computer. It’s under Sharing in System Preferences. Then I login remotely using ssh. I fire up the script, activate Screen Sharing, do my business, shutdown Screen Sharing, log off, and go home.

    Of course, no warranty, no guaranty. Good Luck!

    #!/bin/bash
    
    if [ $# == 0 ]; then
    	echo  "Commands: start, stop, allow <username>, deny <username>"
    	exit
    fi
    
    case "$1" in
    	start)
    		echo 'Starting Remote Access'
    		# Activate Apple Remote Access with current settings
    		sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate
    		;;
    	stop)
    		echo 'Stopping Remote Access'
    		# Deactivate Apple Remote Access
    		sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -deactivate -stop
    		;;
    	allow)
    		if [ -z "$2" ]; then
    			echo  "ERROR: Provide a valid user"
    			exit
    		fi
    		sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -allowAccessFor -specifiedUsers
    		sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -access -on -privs -all -users ${2}
    		;;
    	deny)
    		if [ -z "$2" ]; then
    			echo  "ERROR: Provide a valid user"
    			exit
    		fi  
    		sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -allowAccessFor -specifiedUsers
    		sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -access -off -privs -none -users ${2}  
    		;;  
    esac
    

    I almost forgot. Credit where credit is due! Inspiration from these articles and resources:

    http://www.macosxhints.com/article.php?story=20080318190503111
    http://rentzsch.tumblr.com/post/515009165/starting-vnc-remotely-via-kickstart

     
  • jess 2:15 pm on March 25, 2010 Permalink | Reply
    Tags: Bash, , Tommy Johnson   

    Tommy’s diggall.sh revised 

    Think you know the linux digg command? Ever try to do some simple network discovery with it? This script by Tommy Johnson does some nice condensed discovery. Just pass a domain for the argument.

    ./diggall.sh example.com

    That is all!

    #!/bin/bash
    echo "MX points to:"
    dig MX $1 +short | sort -n
    echo ""
    echo "A points to:"
    dig A $1 +short
    echo ""
    echo "www points to:"
    dig A www.$1 +short
    echo ""
    echo "mail points to:"
    dig A mail.$1 +short
    echo ""
    echo "SOA Nameserver declared:"
    echo `dig SOA $1 +short | awk '{print $1}'`
    echo ""
    #echo "oldmail points to:"
    #dig A oldmail.$1 +short
    domain1=( $(echo $1 | awk -F '.' '{print $1}'))
    domain2=( $(echo $1 | awk -F '.' '{print $2}'))
    domain3=( $(echo $1 | awk -F '.' '{print $3}'))
    domain2=( $(echo $domain2.$domain3 | sed 's/[:.:]$//') )
    first=( $(dig NS +short $domain2 | sed 's/[:.:]$//') )
    second=( $(dig +short NS $domain2 @$first | sed 's/[:.:]$//') )
    #echo "Authoritative nameserver for TLD is: "$second
    #echo '+===============================================+'
    echo "Name Servers as reported by registrar:"
    echo `dig NS $domain1.$domain2 @$second | grep "AUTHORITY SECTION" -A3 | grep -v AUTHORITY | awk '{print $5}'`
    echo "IPs declared by registrar:"
    echo `dig NS $domain1.$domain2 @$second | grep "ADDITIONAL SECTION" -A3 | grep -v ADDITIONAL | awk '{print $5}'`
    #echo '+===============================================+'
    echo ""
    echo "Name Servers in zone file are:"
    dig NS $1 +short
    dig NS $1 | grep "ADDITIONAL SECTION" -A3 | grep -v ADDITIONAL | awk '{print $5}'
    echo "txt record is:"
    dig txt $1 +short
    
     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel