I think that this should fix #49 .

With version 0.12, simple script using net-ssh like following fails with an error

/snip/gems/net-ssh-2.6.5/lib/net/ssh/transport/packet_stream.rb:146:in `enqueue_packet': incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)

require 'rubygems'
require 'net/ssh'
Net::SSH.start('hostname', 'username', :password => 'password') do |ssh|
end

The packet_stream.rb:146 is like

message = encrypted_data + mac

The encoding of encrypted_data is ASCII-8BIT, and encoding of mac is UTF-8. mac is generated at 3 lines above.

mac = client.hmac.digest([client.sequence_number, unencrypted_data].pack("NA*"))

client is an instance of Net::SSH::Transport::HMAC::* (depends algorithm). All those classes are inherited from Net::SSH::Transport::HMAC::Abstract, and Net::SSH::Transport::HMAC::Abstract#digest simply calls OpenSSL::HMAC.digest like:

def digest(data)
  OpenSSL::HMAC.digest(digest_class.new, key, data)[0,mac_length]
end

Thus, I change the encoding of OpenSSL::HMAC.digest return value. Indeed, MRI returns BINARY.

$ macruby -v -ropenssl -e"p OpenSSL::HMAC.digest('sha1', '', '').encoding" 
MacRuby 0.12 (ruby 1.9.2) [universal-darwin10.0, x86_64]
#<Encoding:UTF-8>
$ ruby -v -ropenssl -e"p OpenSSL::HMAC.digest('sha1', '', '').encoding"
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin11.4.0]
#<Encoding:ASCII-8BIT>