FreeBSD Build ProcessDebian Wheezy - UTC or LOCAL hardware clock?

ADSL Router Monitoring

I'd been manually tracking the ADSL router stats for a while before I thought: "Why am I doing this copy and paste stuff when I could automate the whole process?".

The router (a Draytek Vigor) supported telnet command line access, so I needed the expect tool. I did try using netcat (as that is in FreeBSD base), but it killed the router telnet process, which took a reboot to resurrect).

The first attempt used expect and a couple of awk scripts tied together with a shell script. It worked, but all this scattered processing felt wrong. I wondered if I could do it all in python.

There is an expect analog for python - pexpect. Using this and standard python capabilities, the four files of expect, shell and awk scripts could be replaced by one python script. The source code is at the end of this article.

The ADSL line runs at 24Mb (I live opposite the exchange). Here's what the error rate looks like over a 10 day period.

crc cumulative

Figure 1 - Uncorrected Download CRC errors - cumulative total

crc per hour

Figure 2 - Uncorrected Download CRC errors - # per hour

And here's the source code that produces the log file.

  # Extract and log ADSL router stats
  # python [-t n] [-l log_file] [-e email_address]
  # Command arguments:
  #   -t n          Set n as the threshold for the number of crc errors in a time
  #                 period. Error counts above n will be notified via email.  
  #                 Default is 60.
  #   -l log_file   Set pathname of log file.  Default is 
  #                 /home/log/adsl-log.csv
  #    -e address   Set email address to mail for errors in excess of threshold.
  #                 Default is

  import pexpect
  import sys
  import os
  import time
  import getopt
  import re

  def get_adsl_stats():
      "Return  Draytek Vigor adsl stats as a string, using pexpect."
      child = pexpect.spawn("telnet gw")
      child.expect("Password: ")
      child.expect("> ")
      child.sendline("show status")
      child.expect("> ")
      stats = child.before
      return stats

  def extract(field,text_block,now=time.localtime()):
      "Return stringvalue for field in text_block (as presented by Draytek Vigor" 
      "  report). Will also return values for field names date and time."
      if field.lower() == "date":
          return "%4d-%02d-%02d"%(now.tm_year,now.tm_mon,now.tm_mday)
      elif field.lower() == "time":
          return "%02d:%02d:%02d"%(now.tm_hour,now.tm_min,now.tm_sec)
          r ="(\S*)",text_block)
          if r:
          return "(null)"

  def last_line(filename):
      "Returns last line of filename (a string)."
      last = ""
      if os.path.exists(filename):
          for line in open(filename,"r").readlines():
              last = line
      return last

  # list of fields to extract from adsl status
  fields = ("Date", "Time", "System Uptime:", "Up Time:", "Uncorrected Blocks:",\
            "UP Speed:", "Down Speed:", "SNR Margin:", "Loop Att.:", "State:")
  # index for crc count field 
  crc_field = 4 
  # error report threshold (number of crc errors in reporting period)
  threshold = 60
  # log file path name
  log_file = "/home/log/adsl-log.csv"
  # default email address
  email = ""

  # read command line options (if any)
      opts,args = getopt.getopt(sys.argv[1:],'t:l:e:')
      for o,v in opts:
          if o == '-t':
              threshold = int(v)
          elif o == "-l":
              log_file = v
          elif o == "-e":
              email = v
  except getopt.GetoptError,e:
      print >>sys.stderr,"%s: illegal argument: -%s" % (sys.argv[0],e.opt)

  # retrieve adsl stats from router and extract required reporting fields
  t = get_adsl_stats()
  record = list()
  for f in fields:

  # get previous log entry for crc count comparison
  previous_log = last_line(log_file)

  # calculate error count in last period
  crc_now = int(record[crc_field])
  prev_fields = previous_log.split(",")
  # handle case when no previous log entry exists
  if len(prev_fields) > crc_field and prev_fields[crc_field].isdigit():
      crc_prev = int(prev_fields[crc_field])
      crc_cnt = crc_now-crc_prev
      if crc_cnt < 0: crc_cnt = crc_now # assume line reset
      crc_cnt = 0

  # add crc count in last period to record data and write to log file
  l = open(log_file,"a")
  for f in record:

  # check if error report needs to be mailed
  if crc_cnt > threshold:
      subject = "'ADSL Error Rate Alert: %d errors reported.'"%(crc_cnt,)
      # use echo to suppress null body message from mail
      os.system("echo ''|/usr/bin/mail -s "+subject+" "+email)
FreeBSD Build ProcessDebian Wheezy - UTC or LOCAL hardware clock?