Tech Stuff - Ruby
This a collection of notes and an embryonic glossary about the ruby language. Our chosen language for scripting, application and web development since we saw the light in 2003. Mostly it's about the stupid things that you get badly wrong, were confusing (to us) at the beginning, are obscure or just not IOHO adequately covered in the documentation.
Note: Languages should be like hammers. Once you learn to use them they should work the same way every time. We note with considerable regret that ruby has embarked on a fool's errand. The making of an incompatible change from 1.8.x to 1.9.x releases in pursuit of the holy grail of language purity. Good luck with that. While the volume of ruby code is still significantly less than that of Python we note that the Python project cheefullfully accepted a 2 to 3 year transition stratgey in 2007 from Version 2 to the incompatible Version 3. Eight years later (2015) they are still mainaining Version 2 and 3 with, frankly, no prospect of ever escaping. Some of Python's function libraries were written by folks who have long since moved on to new work or may even be dead. Give us a healthy dose of language pragmatism (and we'll learn the few exeptions, perhaps even hold our nose while using them) with consistent backward compatibility rather than purism which means every couple of years stuff which used to work - no longer does. Maybe Ruby can get back its mojo - were it not for Ruby on Rails I doubt it. No one will ever trust them again. Shame, it is (or is that, was) a good language.
Glossary Navigation
Select from the range below:
Symbols ?, #, ::, etc. | a - d | e - l | m - r | s - z
Contents - Notes
Notes: First some notes about our usage and environment:
- We chose ruby because we needed a stand-alone language for both application development and to use as a web scripting language. That left us with Perl (parrot), Python and Ruby. We chose ruby because it appears simpler to add C functionality to speed things up.
- We have decided to completely overhaul our development methodology with ruby as its key component.
- Use ruby as a single very high level language (VHLL) to quickly implement 100% functionality - this is not prototyping this is a fully functional system - it may run like a dog but it lets us move the next stage
- prove (test) system functionality - 100% of functionality. The design - develop feedbackloop occurs at this level
- Use a profiler to find execution hotspots
- Using the profiler output as a priority list - re-implement ONLY THOSE PARTS of the working design (methods or complete objects) into C as required to meet performance objectives
- The operational system will be mixed language implementation with all the top-level or control logic visible in ruby
- Fixes and corrections are applied to the original implementation to keep a fully functional system in pure ruby as well as our hybrid final system.
Well that's the theory. We are currently using it to develop a complete web controlled mail system (SMTP, IMAP, POP3) as a full scale implementation exercise.
- We do development using a windows based HTML Editor-IDE, upload the files using a web page written in ruby and then run them via the same interface. So you'll find these notes very web-centric.
ruby, eruby and erb
The following notes may help when you come to look at the Apache configuration files below:
- modules directly executed by mod_ruby have .rbx or .rb suffixes (the config below allows both) and are assumed to be pure ruby scripts. i.e they are not contained in HTML. These files are located in /usr/local/www/cgi-ruby/ directory (via ScriptAlias directive) and invoked with urls of .../ruby/name.rb or .rbx.
- Ruby scripts embedded in HTML files i.e using <%....%> constructs are contained in files with suffixes of .rhtml and use eruby. These files are located in /usr/local/www/eruby/ directory (via ScriptAlias directive) and are invoked with URLs of ../eruby/myruby.rhtml. A scipt may additionally be placed in the normal directory with a suffix of *.rhtml e.g. index.rhtml - this is less secure since the module content may be viewed.
- Only God seems to know what erb is - certainly google doesn't??
mod_ruby and cookies
To access cookies from mod_ruby:
# writing
request.headers_out['Set-Cookie']="user_id=#{@useridx}/#{@confkey}; path=/"
# reading
request.headers_in['Cookie'].split(';').each do |cookie|
k,v = cookie.split(C_equal)
k.strip!
v.strip!
@cookies[k] = v
end
# or using webrick libraries
require 'webrick/cookie'
#
# misc stuff
#
# Set a cookie
cookie = WEBrick::Cookie.new('user_id','12345')
cookie.expires = Time.now + (86400 * 30)
cookie.path = '/'
request.headers_out['Set-Cookie'] = cookie.to_s
# Read cookies
cookies = WEBrick::Cookie.parse(request.headers_in['Cookie'])
# alternate extend Apache::request with
cookies = request.cookies
request.cookie = cookie
request.cookie = {:key => 'user_id', :value => @user_id, :path =>
'/', :expires => Time.now + 86400 * 30}
Apache Configuration (httpd.conf)
mod_ruby specific fragment of the httpd.conf file to use both ruby and eruby with 'hooked' error 500 response. Note: the fragment below uses FreeBSD LoadModule libexec/ locations for modules Linux RedHat (and we assume other distributions) uses /modules directory.
# Defines Ruby module location
LoadModule ruby_module libexec/apache/mod_ruby.so
# Defines Ruby as Present
AddModule mod_ruby.c
# Allows default file name to include index.rhtml
<IfModule mod_dir.c>
<IfModule mod_php4.c>
<IfModule mod_ruby.c>
DirectoryIndex index.php index.html index.rhtml
</IfModule>
<IfModule !mod_ruby.c>
DirectoryIndex index.php index.html
</IfModule>
</IfModule>
<IfModule !mod_php4.c>
<IfModule mod_ruby.c>
DirectoryIndex index.rhtml index.html
</IfModule>
<IfModule !mod_ruby.c>
DirectoryIndex index.php
</IfModule>
</IfModule>
</IfModule>
# Defines location for .rbx and .rb format ruby files.
ScriptAlias /ruby/ "/usr/local/www/cgi-ruby/"
<Directory "/usr/local/www/cgi-ruby">
AllowOverride None
Options All
Order allow,deny
Allow from all
</Directory>
# Defines cgi location for ruby .rhtml files
ScriptAlias /eruby/ "/usr/local/www/eruby/"
<Directory "/usr/local/www/eruby">
AllowOverride None
Options ExecCGI
Order allow,deny
Allow from all
</Directory>
<IfModule mod_ruby.c>
# Defines features for ruby scripts eg. .rbx files
# for Apache::RubyRun
RubyRequire apache/ruby-run
# exec files under /ruby as ruby scripts.
<Location /ruby>
#defines a ruby script will be used to return error code
# rather than Internal Error 500
ErrorDocument 500 /ruby/error_500.rb
SetHandler ruby-object
RubyHandler Apache::RubyRun.instance
Options ExecCGI
</Location>
# exec *.rbx as ruby scripts.
<Files *.rbx>
SetHandler ruby-object
RubyHandler Apache::RubyRun.instance
</Files>
# exec *.rb as ruby scripts.
<Files *.rb>
SetHandler ruby-object
RubyHandler Apache::RubyRun.instance
</Files>
# Define features for embedded ruby scripts eg. .rhtml files.
# for Apache::ERubyRun
RubyRequire apache/eruby-run
#
# handle files under /eruby as eRuby files by eruby.
<Location /eruby>
#defines a ruby script will be used to return error code
# rather than Internal Error 500
ErrorDocument 500 /ruby/error_500.rb
SetHandler ruby-object
RubyHandler Apache::ERubyRun.instance
Options ExecCGI
</Location>
#
# handle *.rhtml as eruby files.
<Files *.rhtml>
SetHandler ruby-object
RubyHandler Apache::ERubyRun.instance
</Files>
# for Apache::ERbRun
# RubyRequire apache/erb-run
#
# handle files under /erb as eRuby files by ERb.
<Location /erb>
SetHandler ruby-object
RubyHandler Apache::ERbRun.instance
Options ExecCGI
</Location>
# # for debug
# RubyRequire auto-reload
</IfModule>
mod_ruby display Real Error Script
mod_ruby does not fail with warnings AND warning messages are NOT written to the log file (e.g. /var/log/apache/error.log or whatever), only error messages are written to this file. So forget about warnings completely under mod_ruby. Maybe there is a diagnostic level you can set but we have not found it.
When Ruby has a syntax or other error it returns an 'Internal Server Error (500)' and the error message is written to the apache error log. Not very convenient. To return the real error in the response page use the ErrorDocument feature of Apache and the following script (which would be placed in the error_500.rb file referenced in the http.conf file fragment above). Note: If you are using MSIE it intercepts error pages and gives you one of its own - so use a decent browser e.g. any of the Mozilla family):
r = Apache.request
r.content_type = "text/plain"
r.send_http_header
print "Ruby Error: ",r.prev.uri,"\n\n"
print r.prev.error_message
When a error occurs (but not a warning) the actual failure message is displayed on the returned page.
Ruby Objects, Constants and Variables
In Ruby everything is an object? Actually in Ruby we have Objects and references to objects (variables). There are 2 types of Variables. Constant varibles which we'll refer to as constants and regular variables which we'll just call variables.
arr = ["a", "b", "c"] # arr is a variable that has been assigned
# the address of a dynamically created array Object
B = "Hello" # B is a constant that has been assigned
# the address of a dynamically created String Object
Freezing Objects
The idea of freezing an object is to make the object unmodifiable
Example:
arr = [ "a", "b", "c" ]
arr.freeze # This freezes the array object, not the variable
puts arr # output => a, b, c
arr[0] = "3" # This produces an error since the array object is frozen
# we cannot change its contents
Note: But the variable arr can be reassigned to another object
arr = [ "1","2","3"] # This works fine no error here
Constants vs. Variables
All constants must begin with a Capital letter. Reassigning a constant from one object to another gives a warning,
but the reassignment is done anyway.
Bill = "Hello" # Bill is a Constant
bill = "Hello" # bill is a variable
Bill= "Bye" # Gives a Warning
bill = "Bye" # Gives no warning
Double (" " ) vs. Single (' ') Quotes
Enclosing strings in double quotes allows you to use escapings to represent ASCII and Unicode characters. Single quotes allow simple \ escapings only.
puts "bill\n\r" # => bill
puts 'bill\n\r' # => bill\n\r
puts "abcd #{5*3} efg" # => abcd 15 efg
puts 'abcd #{5*3} efg' # => abcd #{5*3} efg
# simple escapings in single strings
puts 'outputs \' single quote'
This one cost serious blood - well the bad cold did not help. IOHO cgi.rb is a bit confusing and inconsistent in behaviour - maybe that explains why there are multiple alternatives.
Depending on how you access the form variables will depend on the results you get.
- An explicit request in 1.8.x of form cgi['myvar'] returns a string
- pre 1.8.x it returns an array
- If you use the form cgi.params it returns a hash
- If your form happens to include file upload (e.g. contains <input type="file"> and an 'enctype="multipart/form-data"') then the fun really starts because:
- if the file size is > 10240 bytes ALL variables are created as Tempfiles
- if < 10240 they are StringIO objects.
- So depending on what is arriving cgi variables can appear as any of a String, StringIO or Tempfile.
# code fragment to test for cgi variable type
require 'cgi'
require 'stringio'
thing = cgi['myvar'].first # gets specific variable
if thing.kind_of? StringIO
puts cgi['myvar'][0].string
# or
puts thing.string #to_s doesn't hack it
elsif thing.kind_of? Tempfile
# get as Tempfile
puts thing.read
elsif thing.kind_of? String
puts thing
else
puts "ERROR:"+thing.class
end
mod_ruby File upload scripts
To upload a file in eruby.
<!-- HTML form fragment -->
<form name='fileupload' enctype="multipart/form-data"
action='/eruby/upload.rhtml' method='post'>
<input type='file' name='myfile' size="40" />
<input type='submit' value"Send it"/>
</form>
# ruby script fragment
require 'cgi'
require 'stringio'
cgi = CGI.new() # New CGI object
# get uri of tx'd file (in tmp normally)
tmpfile = cgi.params['myfile'].first.path
# OR (functionally the same)
tmpfile = cgi.params['myfile'][0].path
# or - this again is the same
tmpfile cgi['myfile'].first.path #first is an alias for [0]
# create a Tempfile reference
fromfile = cgi.params['myfile'].first
#displays the original file name as supplied in the form
puts fromfile.original_filename
# displays the content (mime) type e.g. text/html
puts fromfile.content_type
# create output file reference as original filename in our chosen directory
tofile = '/usr/local/www/somwhere/nice/'+fromfile.original_filename
# copy the file
# note the untaint prevents a security error
# cgi sets up an StringIO object if file < 10240
# or a Tempfile object following works for both
File.open(tofile.untaint, 'w') { |file| file << fromfile.read}
# when the page finishes the Tempfile/StringIO!) thing is deleted automatically
# File transfer forms allow multiple files to be selected and uploaded
# so better code would be
cgi['myfile'].each do |fromfile|
tfn = "/user/local/www/somewhere/else/"+fromfile.original_filename
File.open(tfn.untaint, 'w') { |file| file << fromfile.read}
end
See also some more notes on CGI form variables.
Type Conversion
The following type conversions may be used:
# convert an FixNum to printable format (string)
print 1.to_s # => "1"
# string to integer
print "a".to_i # => 0
print "123".to_i # => 123
print "1a23".to_i # => 1
# integer to float
print 1.to_f # => 1.0
# .hex converts from hex to decimal
print "a".hex # =>10
Iteration and blocks
There are some confusing issues to do with blocks and iterations:
# these both work
myfile.each {|x| x.grep (/me/){|i| puts i+"<br />"} }
myfile.grep (/me/) { |i| puts i+"<br />" }
# this does NOT work - which surprised us
myfile.grep.each (/me/) { |i| puts i+"<br />" }
Files and Directories
Some notes about handling files and directories:
# anything above $SAFE = 0 you need to untaint everything
# directory/file analysis
# prints directories in order followed by ordered files to first level
dirs = "/usr/local/www"+dir
ddirs = Array.new
dfiles = Array.new
md = Dir.open(dirs.untaint).each do |file|
# exclude the crap
if file != "." and file != ".."
# lstat needs full path
if File.lstat(dirs+"/"+file.untaint).directory?
ddirs << file
else
dfiles << file
end
end
end
ddirs.sort.each {|x| puts x+"<br />"}
dfiles.sort.each {|x| puts x+"<br />"}
md.close
# or a single liner
(Dir.entries(/dir/path/)-[".", ".."]).each {|x| print x}
Files Reading 'n writing
Some notes about reading and writing from/to files. Like a lot of folks we like to close files. But ruby can get a little confusing and if its autoclosed then a reference to your file object will throw an exception 'cos is been garbage collected and ..like...its gone man. IO methods don't even open a file so they always close it! Simple rule of thumb. If you handle a file as series of lines there is no autoclose, if you touch it as a file wham its autoclosed see below:
# IO versions have auto open/close
# read whole file into an array
ma = IO.readlines("myfile")
# iterator verion (autoclose)
IO.foreach("myfile") {|line| do something line}
# File treated as lines will not autoclose
# NB: open defaults to mode "r"
mf = File.open("myfile")
# default line separator is $\= RS
mf.each_line {|line| do something}
# slices up input using "end" separator
mf.each_line("end") {|line| do something}
mf.close # gotta close it
# to append data to file (use 'a' or 'a+')
mf = File.open("myfile","a")
mf.print(line)
mf.close
# to add in sequence record - rewrite file
ma = IO.readlines("myfile")
# do stuff in array
mf = File.open("myfile","w")
ma.each {|line| mf.print(line)}
mf.close
# treating as a file (not lines) will autoclose
mf = File.open("myfile") do |f|
f.readlines.each {|l| print l}
end
Stuff about Objects
So you are stuck, the documentation is either non-existent or you can't find it. You have no idea what to do next. The following code provides some information about the object:
# display a list of object methods
thing.methods.each {|x| print x+'<br />'}
# check for a single method
if thing.respond_to?("to_s") then thing.to_s end
# test for a particular class
if thing.kind_of?("String") then ...
# displays objects class
print thing.class.to_s
# you still see thing.type but we understand
# the 'type' method is being deprecated
Seems ruby 1.8 needs the .to_s method to print the class type unlike 1.6.
Ruby Library and Path Locations
This has gotten significantly more complex with gems and a whole bunch of other stuff:
# libary base
[FreeBSD] /usr/local/lib/ruby
[RedHat] /usr/lib/ruby
# assume base for OS above
x.x
# standard library functions
site_ruby/x.x
# site specific including rubygems.rb
site_ruby/x.x/-i386-freebsdx
# architecture specific
gems/x.x/cache
# .gem files for installed gems
gems/x.x/doc
# installed rdoc and ri for installed gems (if any)
gems/x.x/gems
# installed rubygems
gems/x.x/specifications
# .gemspec files for installed gems
The environmental variable RUBYLIB can also be used to control the location and the GLOBAL $: allows manipulation at run-time e.g.:
# adds path/to/my/lib for use in require and load
$: < "path/to/my/lib"
Implementing an each method in a Class
The following ruby code implements an each method where the underlying object already has an each method:
class Thingy
def initialize
@myarray = Array.new
# fill array
@myhash = Hash.new
# fill hash
end
def for_each
@myarray.each {|x| yield x}
end
# for_each with hash
def for_each
@myhash.each {|k,v| yield k,v}
end
end
# usage
t = Thingy.new
# do stuff
# the 'do something is executed for each element
# in our array by the 'yield' in each method
t.for_each {|x| do something x}
# the 'do something is executed for each element
# in our hash by the 'yield' in for_each method
# accessing both k and v
t.for_each {|k,v| do something with k and v}
# you could write a each_key method e.g.
# t.each_key {|k| do something with k}
Using Ruby STD-LIB Resolv.rb
Incomplete documentation for using the fiendishly overcomplicated (and incomplete) Resolv.rb Std-Lib function:
Simple host name look-up:
# returns a string of the IP address
print Resolv.getaddress("www.mydomain.com") # => 192.168.0.1
# NB: must be a full host name
Simple reverse look-up:
# returns a string
print Resolv.getaddress("192.168.0.1")
# gives error - to be investigated
Using the Stub Resolver:
Gets various Resource Records (RR) for the supplied parameter (for description of each RR)
To get the base domain records (NS, SOA and MX) use the following:
# use only the base domain name
# Resolv::DNS::Resource::IN::ANY gets all RR types
dns = Resolv::DNS.new
dns.getresources("mydomain.com", Resolv::DNS::Resource::IN::ANY).collect do |r|
# each record type has different properties - need to test
# see property list below
if r.kind_of?(Resolver::DNS::Resource::IN::MX) then
# Note: every property nned a .to_s method
print r.preference.to_s
print r.exchange.to_s
elsif etc.
...
end
end
To get only the SOA record for the domain:
# use only the base domain name
# Resolv::DNS::Resource::IN::SOA gets only SOA type
dns = Resolv::DNS.new
dns.getresources("mydomain.com", Resolv::DNS::Resource::IN::SOA).collect do |r|
print r.mname
# etc
end
To get only an A record:
# must use the full host name
dns = Resolv::DNS.new
dns.getresources("ftp.mydomain.com", Resolv::DNS::Resource::IN::A).collect do |r|
print r.address
# etc
end
Handling Errors
Resolve throws an exception for everything that moves and gives you only text which is not exactly useful behaviour:
# must use the full host name
dns = Resolv::DNS.new
begin
dns.getresources("ftp.mydomain.com", Resolv::DNS::Resource::IN::A).collect do |r|
print r.address
# etc
end
rescue StandardError => ouch
print ouch
end
Resource Record Properties
Following is an incomplete list of RRs and their properties - excludes boring and not very useful records such as TEXT, HINFO, WKS:
Each RR record type has differing properties
Properties of each record type
must use .to_s on ALL records to convert from type
e.g. obj.address.to_s
MX
preference = preference number
exchange = domain name of mail exchanger
NS
name = name server host name
A
address = ip address
SOA
mname = primary name server for domain
rname = email address (in dot format)
serial = serial number of record
refresh = refresh period
retry = retry
expire = expiration of record
minimum = timeout
PTR
name = host name
Notes on using FileUtils.rb
We needed to change permissions and ownership on a number of files and directories from ruby. FileUtils (up to 1.8.2) does not hack it - but 1.9 FileUtils provides a number of new methods - particularly chown_R, chmod and chmod_R which we wanted. So... We downloaded a CVS copy of FileUtils.rb and tried it. It works perfectly. We now have chown_R etc available in our otherwise stock 1.8.2 ruby lib.
However we ran into a problem. The particular application we were writing takes a command line argument which defines the mode (permission mask) to be applied to a series of directories created by the application e.g. 0770 type values. So we saved the command line argument in a variable and tried to use the variable in the resulting chmod and ruby choked - badly. All the examples shown explicit values being used. Here is out little journey to a solution in code fragments:
# all the class examples use explicit mask values like this - which works
FileUtils.chmod_R(0774,directory)
# when a variable is used ruby chokes - it wants a fixnum
mask = "0774"
directory = "/var/new/some-directory"
FileUtils.chmod_R(mask,directory)
# OK so this really stupid but we wanted to see the result
mask = 0774
directory = "/var/new/some-directory"
FileUtils.chmod_R(mask,directory)
# worked but with wild mask results as expected
# when in doubt use eval so we used this - and it works
mask = "0774"
directory = "/var/new/some-directory"
# messy escaping required
ts = "FileUtils.chmod_R("+mask+',"'+directory+'")'
eval(ts)
Multi-Level Hashes
We use a lot of multi-level hashes to reflect complex record structures. Here are some notes that may help you:
# ruby chokes on this
h = Hash.new # or {}
h[0][0][0] = "one" # gives 'nil' error
# this works
h = Hash.new # or {}
h[0] = {}
h[0][0] = {}
h[0][0][0] = "one"
# or
h = Hash.new # or {}
h[0] = {0=>{0=>"one"}}
# generic rule seems to be add one level at a time on left
Returning Multiple Values
There is no end to the magic of ruby - or more probably we've had too many years with single return procedural languages - OK so you can return structures. Well we're talking ruby now - so forget all that stuff. You can return as many comma separated values as you want with ruby, see examples below:
# simple in-line function with multiple return values
def multi_return(one)
return one+1, one+2, one+3
end
# appears that the above actually returns an array
# e.g. [2,3,4] which can then be assigned or ignored
# assigns all returns
two, three, four = multi_return(one)
# assigns first two returns - ignore third
two,three = multi_return(one)
# by-the-way you can omit return if
# the thing you want to return is the last result
# this works as expected and returns true or false
class Test
def Test.test_it(num)
num > 20
end
end
if Test.test_it(25) then
# do something
end
Errors and Exceptions
We found this very confusing for a long time - still do actually! Here is our explanation - if we are wrong let us know...
The main error object is class Exception, a number of subclasses seem to have been defined - some of which have additional methods or attributes but many don't. The reason they seem to have been created is to classify errors and to allow you to handle, via rescue, specific errors.
Currently avaialable errortypes (subclasses of Exception)
Those marked with * have additional methods
ArgumentError -
IndexError
Interrupt
LoadError
NameError * (name, new, to_s)
NoMemoryError
NoMethodError * (args, new)
NotImplementedError
RangeError
RuntimeError
ScriptError
SecurityError
SignalException
StandardError
SyntaxError
SystemCallError * (===, errno, new)
SystemExit * {new, status)
TypeError
fatal
Methods
The following is an incomplete list of methods. See also Core-API.
# backtrace method
#=================
# the most useful returns array of strings
# usage
=======
# all errors types
begin
...
rescue =>oof
oof.backtrace.each {|x| print x}
end
# OR
# specific error
begin
...
rescue StandardError =>oof
oof.backtrace.each {|x| print x}
end
Roll-your-own
You can make your own error objects to create new methods or provide additional information.
class Someclass
def sm
begin
...
buggy code
...
rescue StandardError =>oops
my Myerror.new("ouch")
my.number(15)
raise my
end
class Myerror < StandardError
attr_reader :num
def initialize
do something
end
def number(num)
@num = num
end
end
end
end
# use
m = Someclass.new
begin
m.sm
rescue Myerror =>oof
print oof.num.to_s
end
Problems, comments, suggestions, corrections (including broken links) or something to add? Please take the time from a busy life to 'mail us' (at top of screen), the webmaster (below) or info-support at zytrax. You will have a warm inner glow for the rest of the day.
