diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 00000000..8e0c0b10 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +ginjo-rfm.gemspec merge=ours +VERSION merge=ours \ No newline at end of file diff --git a/.gitignore b/.gitignore index f732b548..2eba1352 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,21 @@ *.sw? .DS_Store .document +.yardoc local_testing coverage -rdoc +*doc pkg +*.gem +*.bak +*.md.html +Gemfile.lock +*.diff +.bundle/*** +vendor/*** +# you can keep ruby-version in git repo, but don't include it in gem build. +.ruby-version +ruby-prof-*.html +ruby-prof-*.txt +# hide ruby-prof output images (used in html reports) +/*.png \ No newline at end of file diff --git a/.yardopts b/.yardopts new file mode 100644 index 00000000..6b5a54ec --- /dev/null +++ b/.yardopts @@ -0,0 +1,5 @@ +--no-private +- +CHANGELOG.md +LICENSE +lib/rfm/VERSION \ No newline at end of file diff --git a/CHANGELOG b/CHANGELOG deleted file mode 100644 index c06c96a2..00000000 --- a/CHANGELOG +++ /dev/null @@ -1,20 +0,0 @@ -*Lardawge-Rfm 1.4.1* - -* Changed Server#do_action to Server#connect - -* XML Parsing is now done via xpath which significantly speeds up parsing. - -* Changes to accessor method names for Resultset#portals Resultset#fields to Resultset#portal_meta and Resultset#field_meta to better describe what you get back. - -* Added an option to load portal records which defaults to false. This significantly speeds up load time when portals are present on the layout. - - Example: - - result = fm_server('layout').find({:username => "==#{username}"}, {:include_portals => true}) - This will fetch all records with portal records attached. - - result.first.portals would return an empty hash by default. - -* Internal file restructuring. Some classes have changed but it should be nothing a developer would use API wise. Please let me know if it is. - -* Removed Layout#value_lists && Layout#field_controls. Will put back in if the demand is high. Needs a major refactor and different placement if it goes back in. Was broken so it didn't seem to be used by many devs. \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..64173635 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,318 @@ +# Changelog + +## Ginjo-Rfm 3.0.12 + +* Make decimal separator configurable, as FM-Server uses local number settings in result set. + Use configuration option `:decimal_separator`. + +* Fix broken spec for Ox sax parser. + +* Change dependency constraint for rspec to >= v2. + + +## Ginjo-Rfm 3.0.11 + +* Scoping fixes, changes, additions: + + Now takes proc or array of hashes or hash. + + Scope_args for scope proc now defaults to model instance. + + Now handles omits - puts them at end of request array. + +* Support for rom-fmp 0.0.4 - query chaining with compound fm queries. + +* Basic support for portal field writes. + + record['my_relationship::my_field.0'] = 'Adds a new portal record with data for my_field, if auto-create enabled' + record.update_attributes! 'my_relationship::my_field.3' => 'Updates my_field in 3rd portal record, if exists' + +* Fix Base#save! to store raised exceptions in errors, if possible. + +* Fix CompoundQuery to handle nil values in query params. + + +## Ginjo-Rfm 3.0.10 + +* Fixed bug where missing metadata would cause errors when creating/editing records. + +* Added scoping support + + scope = {:person_id => current_user.person_id} + Order.find([{:status => ['open', 'processing']}, {:omit => true, :item_count => "<1"}], :scope => scope) + + class Orders < Rfm::Base + SCOPE = Proc.new { {:expired => "=" } } + end + Order.find({:user_id => '12345'}) + + class Orders < Rfm::Base + SCOPE = Proc.new {|args| {:user_id => args} } + end + Order.find([{:status => ['open', 'processing']}, {:omit => true, :item_count => "<1"}], :scope_args => current_user.id) + +* Code cleanup + +## Ginjo-Rfm 3.0.9 + +* Fixed bug in parser that was appending each portal array recursively to itself. + +* Sax parser template option :template now takes a full-path string. + +* Compatible with dm-filemaker-adapter (for DataMapper ORM). + +* Config now recognizes :template option, allowing alternative parsing templates. + +## Ginjo-Rfm 3.0.8 + +* Implemented proxy option for database connections thru a proxy server. + + config :proxy=>['my.proxy.com', 8888] + +* Implemented erb parsing of config.yml + +* Disabled ```:grammar => :auto``` option. The current xml parser cannot yet use the FMPXMLRESULT grammar for general queries. + + +## Ginjo-Rfm 3.0.7 + +* Changed record creation so that generic records created from non-modelized layouts will be instances of Rfm::Record, instead of instances of a transient model class based on the layout. Transient model classes will foul up serialization and any number of other things. Records created from a user-defined model class will continue to be instances of the model class. + + my_layout.find(12345).class == Rfm::Record + MyModel.find(12345).class == MyModel + + +## Ginjo-Rfm 3.0.6 + +* Fixed duplicate portal-name merging, added specs to test this. +* Minor updates to gem maintenance & release tools. + +## Ginjo-Rfm 3.0.5 + +* Fixed parser handling of `````` element that's missing a `````` element. +* Fixed coercion of repeating field data. +* Fixed case where special characters in Filemaker data yielded array instead of string (sax parsing split text). +* Fixed various bugs in metadata parsing. +* Detached resultset from Rfm::MetaData::Field instance, now attaching only ResultsetMeta to Field. +* Fixed ruby-prof rake task. +* Updated deprecated rspec 2 specs, will now work with rspec 2 or 3. +* Added more specs for some recently found bugs and for sax parser. +* Fixed broken ActiveModel Lint specs in Ruby 2.1. +* General refinements & cleanup. +* Optimizations to sax_parser. +* Fixed typo in field.rb that was causing bugs. + +## Ginjo-Rfm 3.0.4 + +* Corrected reference to @meta in fmpxmllayout.yml. Specs now passing for Layout#load\_layout. +* Added error checking to Layout#load\_layout. +* Fixed setting/saving of repeating fields. Added spec to verify. +* Fixed rspec rake task, fixed rake spec\_multi. +* Changed Ox default parse option to not encode special characters. This is now in tune with other parsers' defaults. + This fixes (among other things) URLs returned from container fields. +* Fixed error in gemspec preventing sax templates from being included in gem build. + +## Ginjo-Rfm 3.0.0 + +* Disabled default port in Connection (was 80), as it was tripping up connections where the port wasn't specified for a :use\_ssl connection on older Rubies. +* Fixes to :ignore\_portals option. +* Removed runtime dependency on activesupport from gemspec. +* Added check in Field#coerce to make sure a '?' is in a string before splitting on '?'. This was breaking repeating container fields. +* Fixed case mismatch in hash key in Factory classes. Added logging of parsing template to logging of parsing backend. +* Fixed bug in field\_control#value\_list. +* Added layout\_meta and resultset\_meta objects. +* Added fmpxmlresult.xml.builder for future use. +* Added Rfm.logger, Rfm.logger=, Config.logger, Config#logger, and config(:logger=>(...)). +* Added logging facility. +* Moved #state method from individual classes to Config class. +* Fixes to Base#update\_attributes. +* Refined multiple :use handling in Config. +* Using rspec 2 +* Removed SubLayout. +* Record.new now automatically creats models based on layout name. Should there be an option to disable this? +* Removed ActiveSupport requirement (of course, ActiveSupport will load if ActiveModle is used, but that is the users' choice). +* Removed XmlMini, XmlParser, and related code & specs. +* Detached resultset from record, so record doesn't drag resultset around with it. +* Disabled automatic model creation from a table-name in a new Rfm::Record when initializing. +* Consolidated Base.new, Base#inititalize into Rfm::Record. +* Fixed validation callbacks issue. +* Fixed: Resultset will politely return [] when asked for non-existent portal\_names. +* Mods to rakefile benchmarking/profiling. +* Refactored Resultset metadata methods. +* Refactored Layout metadata methods. +* Fixed bug in Config#get\_config\_file where a single file path might not be recognized. +* Added connection.rb and moved some methods from Server to Connection. +* Sax parsing rewrite. + +## Ginjo-Rfm 2.1.7 + +* Added field\_mapping awareness to :sort\_field query option. +* Relaxed requirement that query option keys be symbols - can now be strings. + +## Ginjo-Rfm 2.1.6 + +* Fixed typo in Rfm::Record#[]= +* Fixed bug where valid? was called on models without ActiveModel::Validations being loaded. +* Fixed bug where Rfm::Base#reload wasn't clearing mods. + +## Ginjo-Rfm 2.1.5 + +* Fixed bug preventing validation callbacks from running. + +## Ginjo-Rfm 2.1.4 + +* Fixed bug where nil value list would raise exception. + +## Ginjo-Rfm 2.1.3 + +* Fixed bug when loading layout metadata where value lists or field controls with only 1 item would throw an error. + +## Ginjo-Rfm 2.1.2 + +* Fixed config.rb so that :file\_path (to user-defined yml config file) can be specified as a single path string + or as an array of path strings. + +## Ginjo-Rfm 2.1.1 + +* Bug fixes + +* Specs passing in Ruby 1.8.7, 1.9.2. + +## Ginjo-Rfm 2.1.0 + +* Removed ```:include_portals``` query option in favor of ```:ignore_portals```. + +* Added ```:max_portal_rows``` query option. + +* Added field-remapping framework to allow model fields with different names than Filemaker fields. + +* Fix date/time/timestamp translations when writing data to Filemaker. + +* Detached new Server objects from Factory.servers hash, so wont reuse or stack-up servers. + +* Added grammar translation layer between xml parser and Rfm, allowing all supported xml grammars to be used with Rfm. + This will also streamline changes/additions to Filemaker's xml grammar(s). + +* Fixed case statement for ruby 1.9 + +* Configuration ```:use``` option now works for all Rfm objects that respond to ```config```. + +## Ginjo-Rfm 2.0.2 + +* Added configuration parameter ignore\_bad\_data to silence data mismatch errors when loading resultset into records. + +* Added method to load a resultset from file or string. Rfm::Resultset.load\_data(file\_or\_string). + +* Added more specs for the above features and for the XmlParser module. + +## Ginjo-Rfm 2.0.1 + +* Fixed bug in Base.find where options weren't being passed to Layout#find correctly. + +* Fixed bug in rfm.rb when calling #models or #modelize. + +## Ginjo-Rfm 2.0.0 + +* ActiveModel compatibility allows Rails ActiveRecord-style models. + +* Alternative XML parsers using ActiveSupport::XmlMini interface. + +* Compound queries with multiple omitable find-requests. + +* Configuration API manages settings of multiple server/db/layout/etc setups. + +* Full Filemaker metadata support. + +## Ginjo-Rfm 1.4.4 + +* Fixed bug when creating empty value list. + +* Additional fixes for Rfm::VERSION. + +* Fixed Record getter/setter issue. + +* Other minor fixes and cleanup. + +* Added tests to rspec. + +* Documentation cleanup. + +## Ginjo-Rfm 1.4.3 + +* Fixed version management issue. Rfm::VERSION now works. + +## Ginjo-Rfm 1.4.2 + +* Re-implemented: + + Layout#field\_controls + + Layout#value\_lists + +* Enhanced: + + ValueListItem handles both display & data items now. + + Timeout feature from timting (github/timting/rfm). + + Added specs for Record#save. + +* Fixed: + + [Bug] Getting & setting fields with symbol-based keys was producing error. + + [Bug] Setting fields would not update main record hash. + + [Bug] Record#save wasn't merging back into self. + +* Partial Fix: + + server.db.all + db.layout.all + db.script.all + + Note: the "#all" method returns object names (as keys) only. The receiver of the method maintains the full object collection. + + Example: + + server.db.all #=> ['dbname1', 'dbname2', ...] + server.db #=> a DbFactory object (descendant of Hash), containing 0 or more Database objects + +## Lardawge-Rfm 1.4.2 (unreleased) + +* Made nil default on fields with no value. + + Example: + + Old: record.john #=> "" + New: record.john #=> nil + +## Lardawge-Rfm 1.4.1.2 + +* [Bug] Pointing out why testing is soooooo important when refactoring... Found a bug in getter/setter method in Rfm::Record (yes, added spec for it). + +## Lardawge-Rfm 1.4.1.1 + +* [Bug] Inadvertently left out an attr\_reader for server from resultset effecting container urls. + +## Lardawge-Rfm 1.4.1* + +* Changed Server#do\_action to Server#connect. + +* XML Parsing is now done via xpath which significantly speeds up parsing. + +* Changes to accessor method names for Resultset#portals Resultset#fields to Resultset#portal\_meta and Resultset#field\_meta to better describe what you get back. + +* Added an option to load portal records which defaults to false. This significantly speeds up load time when portals are present on the layout. + + Example: + + result = fm_server('layout').find({:username => "==#{username}"}, {:include_portals => true}) + # => This will fetch all records with portal records attached. + + result.first.portals + # => would return an empty hash by default. + +* Internal file restructuring. Some classes have changed but it should be nothing a developer would use API wise. Please let me know if it is. + +* Removed Layout#value\_lists && Layout#field\_controls. Will put back in if the demand is high. Needs a major refactor and different placement if it goes back in. Was broken so it didn't seem to be used by many devs. \ No newline at end of file diff --git a/Gemfile b/Gemfile new file mode 100644 index 00000000..ae557a41 --- /dev/null +++ b/Gemfile @@ -0,0 +1,22 @@ +#!/usr/bin/env ruby + +# Bundler/Gemfile is not necessary to use ginjo-rfm gem. +# It is only here for development & testing. +# See http://bundler.io/man/gemfile.5.html for more info. + +source 'https://rubygems.org' +gemspec + +# If you don't want dev dependencies. +# bundle install --without development + +# Required for ruby < 2.2 +#gem 'activesupport', "< 4" +#gem 'rdoc', "< 6" + +# For ruby 1.8.7 +# gem "activemodel", "~>3" +# gem 'i18n', '0.6.11' +# gem 'nokogiri' , '~>1.5.0' +# gem 'redcarpet', '~>2.3.0' +# gem 'ruby-prof', '~>0.13.0' diff --git a/README.md b/README.md index 348da707..c616a42a 100644 --- a/README.md +++ b/README.md @@ -1,270 +1,616 @@ -# Rfm + -Rdoc location: http://rdoc.info/projects/lardawge/rfm +# ginjo-rfm -## Installation +Rfm is a Ruby-Filemaker adapter, a gem that provides an interface between Filemaker Server and Ruby. Query your Filemaker database, browse result records as persistent objects, and create/update/delete records with a syntax similar to ActiveRecord. Ginjo-rfm picks up from the lardawge-rfm gem and continues to refine code and fix bugs. Version 3 removes the dependency on ActiveSupport and is now a completely independent Gem, able to run most of its core features without requiring any other supporting Gems. ActiveModel features can be activated by adding activemodel to your Gemfile (or requiring activemodel manually). -Terminal: -```bash -gem install lardawge-rfm -``` +## Documentation & Links -Once the gem is installed, you can use rfm in your ruby scripts by requiring it: +* Gem +* Rdoc +* Github +* Discussion +* Original +* Lardawge -```ruby -require 'rubygems' -require 'rfm' -``` -### In Rails >= 3.0 +## Requirements -In the Gemfile: +Ginjo-rfm should run on any machine with a standard ruby installation. Ginjo-rfm's primary function is to interact with Filemaker Server, +however ginjo-rfm does not have to be installed on your Filemaker server - it can be installed on any machine that has network/internet access +to your Filemaker server. -```ruby -gem 'lardawge-rfm' -``` +Ginjo-rfm will work with any Filemaker server that supports the fmresultset.xml grammar over the http protocol. +Since Filemaker Pro client does not support this, Filemaker server is required. Follow Filemaker Server's instructions +for setting up "Custom Web Publishing". -## Connecting +Ginjo-rfm works great with Rails, but it does not require Rails. +You can write simple and powerful stand-alone ruby scripts that use ginjo-rfm to talk to a Filemaker server. + +## Download & Installation + +Find the latest stable release at Rubygems.org. + +In your Gemfile. + + gem 'ginjo-rfm', :require=>'rfm' + +Or manually. + + gem install 'ginjo-rfm' + +You can find the latest development release on github. + + gem 'ginjo-rfm', :git=>'https://github.com/ginjo/rfm.git', :branch=>'master' + +Ginjo-rfm v3 can be run without any other gems, allowing you to create models to interact with your Filemaker servers, layouts, tables, records, and data. If you want the additional features provided by ActiveModel, just add activemodel to your Gemfile (or require it manually). Ginjo-rfm v3 will use the built-in Ruby XML parser REXML by default. If you want to use one of the other supported parsers (libxml-ruby, nokogiri, ox), just add it to your Gemfile or require it manually. If you have several Ruby XML parsers installed, you can specify which one you want Rfm to use by setting the configuration option :parser with one of the supported options (:libxml, :nokogiri, :ox, :rexml). + +Note that while this gem is officially named "ginjo-rfm", you require/load it into your Ruby scripts as simply "rfm". This is in keeping with the original rfm gem from Sixfriedrice and with other forks of the rfm gem. + + + +## Ginjo-rfm Basic Usage + +The first step in getting connected to your Filemaker databases with Rfm (assuming your Filemaker Server is properly set up - see the Filemaker Server instructions for "Custom Web Publishing") is to store your configuration settings in a yaml file or in the RFM_CONFIG hash. The second step is creating a Ruby class (often referred to as a "model" in this documentation) that represent a layout in your Filemaker database. Create as many models as you wish, each pointing to a layout/table-occurrence that you want to work with. The third step is using your new models to query, create, update, and delete records in your Filemaker database. Here's an example setup for a simple order-item table. + +config/rfm.yml + + :host: my.host.com + :account_name: myname + :password: somepass + :database: MyFmDb + +app/models/order\_item.rb + + class OrderItem < Rfm::Base + config :layout => 'order_item_layout' + end + +app/controllers/order\_item\_controller.rb + + def show + @record = OrderItem.find params[:id] + end + + +### Configuration + +In previous versions of Rfm, you may have stored your configuration settings in a variable or constant, then passed those settings to Rfm::Server.new(MY_SETTINGS). Now you can put your configuration settings in a rfm.yml file at the root of your project or in your project's config/ directory, and Rfm will use those settings automatically when building your Model's Server, Database, and Layout objects. + +rfm.yml + + :ssl: true + :timeout: 10 + :port: 443 + :host: my.host.com + :account_name: myname + :password: somepass + :database: MyFmDb + +Or put your configuration settings in a hash called RFM_CONFIG. Rfm will pick those up just as with the yaml file. + + RFM_CONFIG = { + :host => 'my.host.com', + :database => 'MyFmDb', + :account_name => 'myname', + :password => 'somepass', + :ssl => true, + :port => 443, + :timeout => 10 + } + +You can use configuration subgroups to separate global settings from environment-specific settings. + + :ssl: true + :root_cert: false + :timeout: 10 + :port: 443 + :development: + :host: dev.mydomain.com + :account_name: admin + :password: pass + :database: DevFmDb + :production: + :host: live.mydomain.com + :account_name: admin + :password: pass + :database: LiveFmDb + +Then in your environment files (or wherever you put environment-specific configuration in your Ruby project), +specifiy which subgroup to use. + + RFM_CONFIG = {:use => :development} + +You can use configuration subgroups to contain any arbitrary groups of settings. + + :ssl: true + :root_cert: false + :timeout: 10 + :port: 443 + :customer1: + :host: customer1.com + :account_name: cust1 + :password: pass + :database: custOneFmDb + :customer2: + :host: customer2.com + :account_name: cust2 + :password: pass + :database: custTwoFmDb + +Use the configuration setting method `config` to set configuration for specific objects, like Rfm models. When you pass a `:use => :subgroup` to the `config` method, you're saying use that subgroup of settings (on top of any existing upstream configuration). + + class MyModel < Rfm::Base + config :use => :customer1, :layout => 'some_layout' + end + +The current hierarchy of configurable objects in Rfm, starting at the top, is: + +* rfm.yml # file of settings in yaml format +* RFM_CONFIG # user-defined hash +* Rfm::Config # top-level config module, inherits settings from RFM_CONFIG and rfm.yml +* Rfm::Factory # where server, database, and layout objects are managed, inherits settings from Rfm::Config +* Rfm::Base # master modeling class, inherits settings from Rfm::Config +* MyModel # sublcassed custom modeling class, inherits settings from Rfm::Base + +You can also include or extend the Rfm::Config module in any object in your project to gain Rfm configuration abilities for that object. + + module MyModule + include Rfm::Config + config :host => 'myhost.com', :database => 'mydb', :account_name => 'name', :password => 'pass' + # inherits settings from Rfm::Config by default + end + + class Person < Rfm::Base + config :parent => MyModule, :layout => 'some_layout' + # using :parent to set where this object inherits config settings from + end + +Use `get_config` to view the compiled configuration settings for any object. Configuration compilation will start at the top (rfm.yml), then work down the hierarchy of objects to wherever you call the `get_config` method, merging in all global settings along the way. Subgroupings of settings will also be merged, if they are specified in a subgroup filter. A subgroup filter occurs any time you put `:use => :subgroup` in your configuration setting. You can have multiple subgroup filters, and when configuration compilation occurs, all subgroup filters are stacked up into an array and processed in order (as if you typed `:use=>[:subgroup1, :subgroup2, subgroup3, ...]` which is also allowed). `get_config` returns a compiled configuration hash, leaving all configuration settings in all modules and classes un-touched. + + Person.get_config + + # => {:ssl => true, :timeout => 10, :root_cert => false, :port => 443, + :host => 'myhost', :database => 'mydb', :layout => 'some_layout', + :account_name => 'name', :password => 'pass' + } + +#### Configuration Options + +Following are all of the recognized configuration options, including defaults if applicable. +See `Rfm::Config::CONFIG_KEYS` for a list of currently allowed configuration options. + + :host => 'localhost' + :port + :ssl => true + :root_cert => true + :root_cert_name => '' + :root_cert_path => '/' + :account_name => '' + :password => '' + :proxy => false # Pass an array of Net::HTTP::Proxy options (p_addr, p_port = nil, p_user = nil, p_pass = nil). + :log_actions => false + :log_responses => false + :log_parser => false + :warn_on_redirect => true + :raise_on_401 => false + :timeout => 60 + + :use # Use configuration subgroups, or filter configuration subgoups. + :layout # Specify the name of the layout to use. + :parent => 'Rfm::Config' # The parent configuration object of the current configuration object, as string. + :file_name => 'rfm.yml # Name of configuration file to load yaml from. + :file_path => ['', 'config/'] # Array of additional file paths to look for configuration file. + :parser # Prefferred XML parser. Can be :libxml, :nokogiri, :ox, :rexml. + # You must also require the parsing gem or specify it in your gemfile, + # if not using the built-in Ruby XML parser REXML. + # You only need to use this option if you have multiple + # parsing gems loaded and want to use a specfic one. + # Otherwise, Rfm will use the best parser it can find amongst your currently loaded parsing gems. + :ignore_bad_data => nil # Instruct Rfm to ignore data mismatch errors when loading a resultset. + :decimal_separator => '.' # FileMaker uses local number format of the server. It stores input as text. + # So on european servers a number value of '1.200,50' is considered valid and interpreted as 1200.5 + # To support this, :decimal_separator can be configured and any other characters than digits and the separator are ignored + +### Using Models + +Rfm models provide easy access, modeling, and persistence of your Filemaker data. A ginjo-rfm model is basically an alias to a specific layout in your Filemaker database and provides all of the query options found in a classic rfm layout object. The model and/or the layout object is where you do most of your work with rfm. For more details about what methods and options are available to a model or layout object, see the documentation for the {Rfm::Layout} and {Rfm::Base} classes. + + class User < Rfm::Base + config :layout => 'my_user_layout' + attr_accessor :password + end + + @user = User.new(:login => 'bill', :password => 'xxxxxxxx', :email => 'my@email.com') + @user.encrypt_password + @user.save! + + @user.record_id + # => '12345' + + @user.field_names + # => ['login', 'encryptedPassword', 'email', 'groups', 'lastLogin' ] + + User.total_count + # => 35467 + + @user = User.find 12345 + @user.update_attributes(:login => 'william', :email => 'myother@email.com') + @user.save! + +If using Rails, put your model code in files within your models/ directory. + +app/models/user.rb + + class User < Rfm::Base + config :layout => 'user_layout' + end + +If you prefer, you can create models on-the-fly from any layout. + + my_rfm_layout_object.modelize + + # => MyLayoutName (subclassed from Rfm::Base, represented by your layout's name) + +Or create models for an entire database, all at once. + + Rfm.modelize /_xml/i, 'my_database', :my_config_group + + # => [MyLayoutXml, AnotherLayoutXml, ThirdLayoutXml, AndSoOnXml, ...] + # The regex in the first parameter is optional and filters the layout names in the specified database. + # Omit the regex parameter to modelize all possible layouts in the specified database (careful with this one!). + +With ActiveModel loaded, you get callbacks, validations, errors, serialization, and a handful of other features extracted from Rails ActiveRecord. Not all ActiveModel features are supported (yet) in ginjo-rfm, but adapters can be hand-rolled in the meantime. + +In your Gemfile + + gem 'activemodel' + +Or without Bundler + + require 'active_model' + +Then use ActiveModel features in your Rfm models + + class MyModel < Rfm::Base + before_create :encrypt_password + after_validate "puts 'yay!'" + validates :email, :presence => true + end + + @my_model = MyModel.new + @my_model.valid? + @my_model.save! + @my_model.errors + +To learn more about ActiveModel, see the ActiveModel or RubyOnRails documentation: + +* +* +* + +Once you have an Rfm model or layout, you can use any of the standard Rfm commands to create, search, edit, and delete records. To learn more about these commands, see below for Databases, Layouts, Resultsets, and Records. Or checkout the API documentation for Rfm::Server, Rfm::Database, Rfm::Layout, Rfm::Record, and Rfm::Base. + +#### Two Small Changes in Rfm Return Values + +(From Rfm v2 onward) When using Models to retrieve records using the `any` method or the `find(record_id)` method, the return values will be single Rfm::Record objects. This differs from the original Rfm behavior of these methods when accessed directly from the the Rfm::Layout instance, where the return value is always a Rfm::Resultset. + + MyModel.find(record_id) == my_layout.find(record_id)[0] + MyModel.any == my_layout.any[0] + + +### Getting Rfm Server, Database, and Layout Objects Manually + +Well... not entirely manually. To get server, db, and layout objects as in previous versions of Rfm, see the section "Working with classic Rfm features". Newer versions of ginjo-rfm can use the configuration options to build these objects. + +Create a layout object using default configuration settings. + + my_layout = Rfm.layout 'layout_name' + +Create a layout object using a subgroup of configuration settings. + + my_layout = Rfm.layout :subgroup_name + +Create a layout object passing in a layout name, multiple config subgroups to merge, and specific settings. + + my_layout = Rfm.layout 'layout_name', :other_server, :log_actions => true + +The same can be done for servers and databases. + + my_server = Rfm.server 'my.host.com' + my_database = Rfm.database :development, :ssl => false, :root_cert => false + my_database = Rfm.db :production + # db and database are interchangeable aliases in Ginjo-rfm 2.0 + +You can query your Filemaker objects for the familiar meta-data. + + my_server.databases.all.names + my_server.databases['MyFmDb'] + my_database.layouts + my_layout.value_lists + my_layout.field_names + my_layout.portal_meta + +Here are two new fun Layout methods: + + my_layout.total_count # => total records in table + my_layout.count(:some_field => 'search criteria', ...) # Returns foundset_count only, no records. + +See the API documentation for the lowdown on new methods in Rfm Server, Database, and Layout objects. + +### Shortcuts, Tips & Tricks + +All Rfm methods that take a configuration hash have two possible shortcuts. + +(1) If you pass a symbol before the hash, it is interpreted as subgroup specification or subgroup filter + + config :mygroup, :layout => 'mylayout' + # This will add the following configuration options to the object you called 'config' on. + # :use => :mygroup, :layout => 'mylayout' + + get_config :othergroup + # This will return global configuration options merged with configuration options from :othergroup. + # :use => [:mygroup, :othergroup], :layout => 'mylayout' + +(2) If you pass a string before any symbols or hashes, it is interpreted as one of several possible configuration settings - usually a layout name, a database name, or a server hostname. The interpretation is dependent on the method being called. Not all methods will make use of a string parameter. + + class MyModel < Rfm::Base + config 'MyLayoutName' + # In this context, this is the same as + # config :layout => 'MyLayoutName' + end + + Rfm.database 'MyDatabaseName' + # In this context, this is the same as + # Rfm.database :database => 'MyDatabaseName' + + Rfm.modelize 'MyDatabaseName', :group1 + # In this context, this is the same as + # Rfm.modelize :database => 'MyDatabaseName', :use => :group1 + +Just about anything you can do with a Rfm layout, you can also do with a Rfm model. + + MyModel.total_count + MyModel.field_names + MyModel.database.name + +There are a number of methods within Rfm that have been made accessible from the top-level Rfm module. Note that the server/database/layout methods are new to Rfm and are not the same as Rfm::Server, Rfm::Database, and Rfm::Layout. See the above section on "Getting Rfm Server, Database, and Layout Objects Manually" for an overview of how to use the new server/database/layout methods. + + # Any of these methods can be accessed via Rfm. + + Rfm::Factory :servers, :server, :db, :database, :layout, :models, :modelize + Rfm::Config :config, :get_config, :config_clear + Rfm::Resultset :ignore_bad_data + Rfm::SaxParser :backend + +If you are working with a Filemaker database that returns codes like '?' for a missing value in a date field, Rfm will throw an error. Set your main configuration, your server, or your layout to `ignore_bad_data true`, if you want Rfm to silently ignore data mismatch errors when loading resultset data. If ActiveRecord is loaded, and your resultset is loaded into a Rfm model, your model records will log these errors in the @errors attribute. + + Rfm.config :ignore_bad_data => true + + class MyModel < Rfm::Base + config 'my_layout' + end + + result = MyModel.find(:name => 'mike') + # Assuming the Filemaker field 'some_date_field' contains a bad date value '?' + result[0].errors.full_messages + # ['some_date_field invalid date'] + + # To be more specific about what objects you want to ignore data errors + MyModel.layout.ignore_bad_data true + +## Working with "Classic" Rfm Features + +All of Rfm's original features and functions are available as they were before, though some low-level functionality has changed slightly. See the documentation for each module & class for the specifics on low-level methods and functionality. + + +### Connecting IMPORTANT:SSL and Certificate verification are on by default. Please see Server#new in rdocs for explanation and setup. You connect with the Rfm::Server object. This little buddy will be your window into FileMaker data. -```ruby -require 'rfm/rfm' + require 'rfm' -my_server = Rfm::Server.new( - :host => 'myservername', - :username => 'user', - :password => 'pw', - :ssl => false -) -``` + my_server = Rfm::Server.new( + :host => 'myservername', + :account_name => 'user', + :password => 'pw', + :ssl => false + ) if your web publishing engine runs on a port other than 80, you can provide the port number as well: -```ruby -my_server = Rfm::Server.new( - :host => 'myservername', - :username => 'user', - :password => 'pw', - :port => 8080, - :ssl => false -) -``` + my_server = Rfm::Server.new( + :host => 'myservername', + :account_name => 'user', + :password => 'pw', + :port => 8080, + :ssl => false, + :root_cert => false + ) -## Databases and Layouts +### Databases and Layouts All access to data in FileMaker's XML interface is done through layouts, and layouts live in databases. The Rfm::Server object has a collection of databases called 'db'. So to get ahold of a database called "My Database", you can do this: -```ruby -my_db = my_server.db["My Database"] -``` + my_db = my_server.db["My Database"] As a convenience, you can do this too: -```ruby -my_db = my_server["My Database"] -``` + my_db = my_server["My Database"] Finally, if you want to introspect the server and find out what databases are available, you can do this: -```ruby -all_dbs = my_server.db.all -``` + all_dbs = my_server.db.all In any case, you get back Rfm::Database objects. A database object in turn has a property called "layout": -```ruby -my_layout = my_db.layout["My Layout"] -``` + my_layout = my_db.layout["My Layout"] Again, for convenience: -```ruby -my_layout = my_db["My Layout"] -``` + my_layout = my_db["My Layout"] And to get them all: -```ruby -all_layouts = my_db.layout.all -``` + all_layouts = my_db.layout.all Bringing it all together, you can do this to go straight from a server to a specific layout: -```ruby -my_layout = my_server["My Database"]["My Layout"] -``` + my_layout = my_server["My Database"]["My Layout"] -## Working with Layouts +### Working with Layouts Once you have a layout object, you can start doing some real work. To get every record from the layout: -```ruby -my_layout.all # be careful with this -``` + my_layout.all # be careful with this To get a random record: -```ruby -my_layout.any -``` + my_layout.any To find every record with "Arizona" in the "State" field: -```ruby -my_layout.find({"State" => "Arizona"}) -``` + my_layout.find({"State" => "Arizona"}) To add a new record with my personal info: -```ruby -my_layout.create({ - :first_name => "Geoff", - :last_name => "Coffey", - :email => "gwcoffey@gmail.com"} -) -``` + my_layout.create({ + :first_name => "Geoff", + :last_name => "Coffey", + :email => "gwcoffey@gmail.com"} + ) Notice that in this case I used symbols instead of strings for the hash keys. The API will accept either form, so if your field names don't have whitespace or punctuation, you might prefer the symbol notation. -To edit the record whos recid (filemaker internal record id) is 200: +To edit the record whose recid (filemaker internal record id) is 200: -```ruby -my_layout.edit(200, {:first_name => 'Mamie'}) -``` + my_layout.edit(200, {:first_name => 'Mamie'}) Note: See the "Record Objects" section below for more on editing records. To delete the record whose recid is 200: -```ruby -my_layout.delete(200) -``` + my_layout.delete(200) -All of these methods return an Rfm::Result::ResultSet object (see below), and every one of them takes an optional parameter (the very last one) with additional options. For example, to find just a page full of records, you can do this: +All of these methods return an Rfm::Resultset object (see below), and every one of them takes an optional parameter (the very last one) with additional options. For example, to find just a page full of records, you can do this: -```ruby -my_layout.find({:state => "AZ"}, {:max_records => 10, :skip_records => 100}) -``` + my_layout.find({:state => "AZ"}, {:max_records => 10, :skip_records => 100}) -For a complete list of the available options, see the "expand_options" method in the Rfm::Server object in the file named rfm_command.rb. +For a complete list of the available options, see the "Common Options" section in the layout.rb file. -Finally, if filemaker returns an error when executing any of these methods, an error will be raised in your ruby script. There is one exception to this, though. If a find results in no records being found (FileMaker error # 401) I just ignore it and return you a ResultSet with zero records in it. If you prefer an error in this case, add :raise_on_401 => true to the options you pass the Rfm::Server when you create it. +Finally, if filemaker returns an error when executing any of these methods, an error will be raised in your Ruby script. There is one exception to this, though. If a find results in no records being found (FileMaker error # 401) I just ignore it and return you a Resultset with zero records in it. If you prefer an error in this case, add :raise_on_401 => true to the options you pass the Rfm::Server when you create it. -## ResultSet and Record Objects +### Resultset and Record Objects -Any method on the Layout object that returns data will return a ResultSet object. Rfm::Result::ResultSet is a subclass of Array, so first and foremost, you can use it like any other array: +Any method on the Layout object that returns data will return a Resultset object. Rfm::Resultset is a subclass of Array, so first and foremost, you can use it like any other array: -```ruby -my_result = my_layout.any -my_result.size # returns '1' -my_result[0] # returns the first record (an Rfm::Result::Record object) -``` + my_result = my_layout.any + my_result.size # returns '1' + my_result[0] # returns the first record (an Rfm::Record object) -The ResultSet object also tells you information about the fields and portals in the result. ResultSet#fields and ResultSet#portals are both standard ruby hashes, with strings for keys. The fields hash has Rfm::Result::Field objects for values. The portals hash has another hash for its values. This nested hash is the fields on the portal. This would print out all the field names: +The Resultset object also tells you information about the fields and portals in the result. Resultset#field\_meta and Resultset#portal\_meta are both standard Ruby hashes, with strings for keys. The fields hash has Rfm::Metadata::Field objects for values. The portals hash has another hash for its values. This nested hash is the fields on the portal. This would print out all the field names: -```ruby -my_result.fields.each { |name, field| puts name } -``` + my_result.field_meta.each { |name, field| puts name } + +Or, as a convenience, you can do this: + + my_result.field_names This would print out the tables each portal on the layout is associated with. Below each table name, and indented, it will print the names of all the fields on each portal. -```ruby -my_result.portals.each { |table, fields| - puts "table: #{table}" - fields.each { |name, field| puts "\t#{name}"} -} -``` + my_result.portals.each { |table, fields| + puts "table: #{table}" + fields.each { |name, field| puts "\t#{name}"} + } + +Also as a convenience, you can do this: + + my_result.portal_names -But most importantly, the ResultSet contains record objects. Rfm::Result::Record is a subclass of Hash, so it can be used in many standard ways. This code would print the value in the 'first_name' field in the first record of the ResultSet: +But most importantly, the Resultset contains record objects. Rfm::Record is a subclass of Hash, so it can be used in many standard ways. This code would print the value in the 'first_name' field in the first record of the Resultset: -```ruby -my_record = my_result[0] -puts my_record["first_name"] -``` + my_record = my_result[0] + puts my_record["first_name"] -As a convenience, if your field names are valid ruby method names (ie, they don't have spaces or odd punctuation in them), you can do this instead: +As a convenience, if your field names are valid Ruby method names (ie, they don't have spaces or odd punctuation in them), you can do this instead: -```ruby -puts my_record.first_name -``` + puts my_record.first_name -Since ResultSets are arrays and Records are hashes, you can take advantage of Ruby's wonderful expressiveness. For example, to get a comma-separated list of the full names of all the people in California, you could do this: +Since Resultsets are arrays and Records are hashes, you can take advantage of Ruby's wonderful expressiveness. For example, to get a comma-separated list of the full names of all the people in California, you could do this: -```ruby -my_layout.find(:state => 'CA').collect {|rec| "#{rec.first_name} #{rec.last_name}"}.join(", ") -``` + my_layout.find(:state => 'CA').collect {|rec| "#{rec.first_name} #{rec.last_name}"}.join(", ") Record objects can also be edited: -```ruby -my_record.first_name = 'Isabel' -``` + my_record.first_name = 'Isabel' Once you have made a series of edits, you can save them back to the database like this: -```ruby -my_record.save -``` + my_record.save The save operation causes the record to be reloaded from the database, so any changes that have been made outside your script will also be picked up after the save. If you want to detect concurrent modification, you can do this instead: -```ruby -my_record.save_if_not_modified -``` + my_record.save_if_not_modified This version will refuse to update the database and raise an error if the record was modified after it was loaded but before it was saved. -Record objects also have portals. While the portals in a ResultSet tell you about the tables and fields the portals show, the portals in a Record have the actual data. For example, if an Order record has Line Item records, you could do this: +Record objects also have portals. While the portals in a Resultset tell you about the tables and fields the portals show, the portals in a Record have the actual data. For example, if an Order record has Line Item records, you could do this: -```ruby -my_order = order_layout.any[0] # the [0] is important! -my_lines = my_order.portals["Line Items"] -``` + my_order = order_layout.any[0] # the [0] is important! + my_lines = my_order.portals["Line Items"] At the end of the previous block of code, my_lines is an array of Record objects. In this case, they are the records in the "Line Items" portal for the particular order record. You can then operate on them as you would any other record. NOTE: Fields on a portal have the table name and the "::" stripped off of their names if they belong to the table the portal is tied to. In other words, if our "Line Items" portal includes a quantity field and a price field, you would do this: -```ruby -my_lines[0]["Quantity"] -my_lines[0]["Price"] -``` + my_lines[0]["Quantity"] + my_lines[0]["Price"] You would NOT do this: -```ruby -my_lines[0]["Line Items::Quantity"] -my_lines[0]["Line Items::Quantity"] -``` + my_lines[0]["Line Items::Quantity"] + my_lines[0]["Line Items::Quantity"] My feeling is that the table name is redundant and cumbersome if it is the same as the portal's table. This is also up for debate. Again, you can string things together with Ruby. This will calculate the total dollar amount of the order: -```ruby -total = 0.0 -my_order.portals["Line Items"].each {|line| total += line.quantity * line.price} -``` + total = 0.0 + my_order.portals["Line Items"].each {|line| total += line.quantity * line.price} -## Data Types +### Data Types FileMaker's field types are coerced to Ruby types thusly: - Text Field -> String object - Number Field -> BigDecimal object # see below - Date Field -> Date object - Time Field -> DateTime object # see below - TimeStamp Field -> DateTime object - Container Field -> URI object + Text Field -> String object + Number Field -> BigDecimal object # see below + Date Field -> Date object + Time Field -> DateTime object # see below + TimeStamp Field -> DateTime object + Container Field -> URI object -FileMaker's number field is insanely robust. The only data type in ruby that can handle the same magnitude and precision of a FileMaker number is Ruby's BigDecimal. (This is an extension class, so you have to require 'bigdecimal' to use it yourself). Unfortuantely, BigDecimal is not a "normal" ruby numeric class, so it might be really annoying that your tiny filemaker numbers have to go this route. This is a great topic for debate. +FileMaker's number field is insanely robust. The only data type in Ruby that can handle the same magnitude and precision of a FileMaker number is Ruby's BigDecimal. (This is an extension class, so you have to require 'bigdecimal' to use it yourself). Unfortuantely, BigDecimal is not a "normal" Ruby numeric class, so it might be really annoying that your tiny filemaker numbers have to go this route. This is a great topic for debate. -Also, Ruby doesn't have a Time type that stores just a normal time (with no date attached). The Time class in ruby is a lot like DateTime, or a Timestamp in FileMaker. When I get a Time field from FileMaker, I turn it into a DateTime object, and set its date to the oldest date Ruby supports. You can still compare these in all the normal ways, so this should be fine, but it will look weird if you, ie, to_s one and see an odd date attached to your time. +Also, Ruby doesn't have a Time type that stores just a normal time (with no date attached). The Time class in Ruby is a lot like DateTime, or a Timestamp in FileMaker. When I get a Time field from FileMaker, I turn it into a DateTime object, and set its date to the oldest date Ruby supports. You can still compare these in all the normal ways, so this should be fine, but it will look weird if you, ie, to_s one and see an odd date attached to your time. Finally, container fields will come back as URI objects. You can: @@ -274,7 +620,7 @@ Finally, container fields will come back as URI objects. You can: Specifically, the URI refers to the _contents_ of the container field. When accessed, the file, picture, or movie in the field will be downloaded. -## Troubleshooting +### Troubleshooting There are two cheesy methods to help track down problems. When you create a server object, you can provide two additional optional parameters: @@ -286,16 +632,238 @@ When this is 'true' your script will dump the actual response it got from FileMa So, for an annoying, but detailed load of output, make a connection like this: -```ruby -my_server # Rfm::Server.new( - :host #> 'myservername', - :username #> 'user', - :password #> 'pw', - :log_actions #> true, - :log_responses #> true -) -``` + my_server => Rfm::Server.new( + :host => 'myservername', + :account_name => 'user', + :password => 'pw', + :log_actions => true, + :log_responses => true + ) + +### Source Code + +If you were tracking ginjo-rfm on github before the switch to version 2.0.0, please accept my humblest apologies for making a mess of the branching. The pre 2.0.0 edge branch has become master, and the pre 2.0.0 master branch has become ginjo-1-4-stable. I don't intend to make that kind of hard reset again, at least not on public branches. Master will be the branch to find the latest-greatest public source, and 'stable' branches will emerge as necessary to preserve historical releases. + +### Still To Do + +Repeating field compatibility, more coverage of Filemaker's query syntax, more error classes, more specs, and more documentation. + + + + +## Version Highlights + +### Version 3.0 + +There are many changes in version 3, but most of them are under the hood. Here are some highlights. + +* Compatibility with Ruby 2.1.2 (and 2.0.0, 1.9.3, 1.8.7). + +* XML parsing rewrite. +The entire XML parsing engine of Rfm has been rewritten to use only the sax/stream parsing schemes of the supported Ruby XML parsers (libxml-ruby, nokogiri, ox, rexml). There were two main goals in this rewrite: 1, to separate the xml parsing code from the Rfm/Filemaker objects, and 2, to remove the hard dependency on ActiveSupport. See below for parsing configuration options. + +* Better logging capabilities. +Added Rfm.logger, Rfm.logger=, Config.logger, Config#logger, and config(:logger=>(...)). + +* Added field-mapping awareness to :sort_field query option. + +* Relaxed requirement that query option keys be symbols - can now be strings or symbols. + +* Detached resultset from record, so record doesn't drag resultset around with it. + +* Bug fixes and refinements in modeling, configuration, metadata access, and Rfm object instantiation. + +See the changelog or the commit history for more details on changes in ginjo-rfm v3. + +### Version 2.1 + +* Portals are now included by default. + Removed `:include_portals` query option in favor of `:ignore_portals`. + Added `:max_portal_rows` query option. +* Added field-remapping framework to allow model fields with different names than Filemaker fields. + + class User < Rfm::Base + config :field_mapping => { + # => + 'userName' => 'login', + 'First Name' => 'first_name', + 'Last Name' => 'last_name', + 'IDperson' => 'person_id' + } + end + + User.find(:login=>'bill') # => [{'login' => 'bill', 'first_name' => 'Bill', ...}, ...] + +* Fixed date/time/timestamp translations when writing data to Filemaker. +* Detached new Server objects from Factory.servers hash, so wont reuse or stack-up servers. +* Added grammar translation layer between xml parser and Rfm, allowing all supported xml grammars to be used with Rfm. + This will also streamline changes/additions to Filemaker's xml grammar(s). +* Added ability to manually import fmpresultset and fmpxmlresult data (from file, variable, etc.). + + Rfm::Resultset.load_data(file_or_string). + +* Compatibility fixes for Ruby 1.9. +* Configuration `:use` option now works for all Rfm objects that respond to `config`. + + +### Version 2.0 + +* Rails-like modeling with ActiveModel +* Support for multiple XML Parsers +* Configuration API +* Compound Filemaker queries with omitable requests +* Full metadata support + + +#### Data Modeling with ActiveModel + +If you can load ActiveModel in your project, you can have model callbacks, validations, and other ActiveModel features. +If you can't load ActiveModel (because you're using something incompatible, like Rails 2), +you can still use Rfm models... minus the ActiveModel-specific features like callbacks and validations. Rfm models give you basic +data modeling with easy configuration and CRUD features. + + class User < Rfm::Base + config :layout=>'user_layout' + before_save :encrypt_password + validate :valid_email_address + end + + @user = User.new :username => 'bill', :password => 'pass' + @user.email = 'my@email.com' + @user.save! + + +#### Choice of XML Parsers + +Note that this section only applies to ginjo-rfm v2. See notes for ginjo-rfm v3 for v3 parsing options. + +Ginjo-rfm 2.0 uses ActiveSupport's XmlMini parsing interface, which has built-in support for +LibXML, Nokogiri, and REXML. Additionally, ginjo-rfm includes adapters for Ox and Hpricot parsing. +You can specifiy which parser to use or let Rfm decide. + + Rfm.config :parser => :libxml + +If you're not able to install one of the faster parsers, ginjo-rfm will fall back to +Ruby's built-in REXML. Want to roll your own XML adapter? Just pass it to Rfm as a module. + + Rfm.config :parser => MyHomeGrownAdapter + +Choose your preferred parser globaly, as in the above example, or set a different parser for each model. + + class Order < Rfm::Base + config :parser => :hpricot + end + +The current parsing options are + + :jdom -> JDOM (for JRuby) + :oxsax -> Ox SAX + :libxml -> LibXML Tree + :libxmlsax -> LibXML SAX + :nokogirisax -> Nokogiri SAX + :nokogiri -> Nokogiri Tree + :hpricot -> Hpricot Tree + :rexml -> REXML Tree + :rexmlsax -> REXML SAX + + +#### Configuration API + +The ginjo-rfm configuration module lets you store your settings in several different ways. Store some, or all, of your project-specific settings in a rfm.yml file at the root of your project, or in your Rails config/ directory. Settings can also be put in a RFM_CONFIG constant at the top level of your project. Configuration settings can be simple key=>values, or they can be named groups of key=>values. Configuration can also be passed to various Rfm methods during load and runtime, as individual settings or as groups. + +rfm.yml + + :ssl: true + :root_cert: false + :timeout: 10 + :port: 443 + :host: live.mydomain.com + :account_name: admin + :password: pass + :database: MyFmDb + +Set a model's configuration. + + class MyModel < Rfm::Base + config :layout => 'mylayout' + end + + +#### Compound Filemaker Queries, with Omitable FMP Find Requests + +Create a Filemaker 'omit' request by including an :omit key with a value of true. + + my_layout.find :field1 => 'val1', :field2 => 'val2', :omit => true + +Create multiple Filemaker find requests by passing an array of hashes to the #find method. + + my_layout.find [{:field1 => 'bill', :field2 => 'admin'}, {:field2 => 'staff', :field3 => 'inactive', :omit => true}, ...] + +If the value of a field in a find request is an array of strings, the string values will be logically OR'd in the query. + + my_layout.find :fieldOne => ['bill','mike','bob'], :fieldTwo =>'staff' + + +#### Full Metadata Support + +* Server databases +* Database layouts +* Database scripts +* Layout fields +* Layout portals +* Resultset meta +* Field definition meta +* Portal definition meta + +There are also many enhancements to make it easier to get the objects or data you want. Some examples: + +Get a database object using default config + + Rfm.db 'my_db' + +Get a layout object using config grouping :my_group + + Rfm.layout :my_group + +Get the total count of all records in the table + + MyModel.total_count + +Get the portal names (table-occurrence names) on the current layout + + MyModel.portal_names + +Get the names of fields on the current layout + + my_record.field_names + + +### From Version 1.4.x + +From ginjo-rfm 1.4.x, the following features are also included. + +Connection timeout settings + + Rfm.config :timeout => 10 + +Value-list alternate display + + i = array_of_value_list_items[3] # => '8765' + i.value # => '8765' + i.display # => '8765 Amy' + + + +## Credits + +Rfm was primarily designed by Six Fried Rice co-founder Geoff Coffey. + +Other lead contributors: + +* Mufaddal Khumri helped architect Rfm in the most Ruby-like way possible. He also contributed the outstanding error handling code and a comprehensive hierarchy of error classes. +* Atsushi Matsuo was an early Rfm tester, and provided outstanding feedback, critical code fixes, and a lot of web exposure. +* Jesse Antunes helped ensure that Rfm is stable and functional. +* Larry Sprock added ssl support, switched the xml parser to a much faster Nokogiri, added the rspec testing framework, and refined code architecture. +* William Richardson is the current maintainer of the ginjo-rfm fork and added support for multiple xml parsers, ActiveModel integration, field mapping, compound queries, logging, scoping, and a configuration framework. -## Copyright -Copyright (c) 2007 Six Fried Rice, LLC and Mufaddal Khumr. See LICENSE for details. diff --git a/Rakefile b/Rakefile index a1b64bab..85a4c393 100644 --- a/Rakefile +++ b/Rakefile @@ -1,50 +1,170 @@ require 'rubygems' require 'rake' +require './lib/rfm' -begin - require 'jeweler' - Jeweler::Tasks.new do |gem| - gem.name = "lardawge-rfm" - gem.summary = "Ruby to Filemaker adapter" - gem.description = "Rfm brings your FileMaker data to Ruby. Now your Ruby scripts and Rails applications can talk directly to your FileMaker server." - gem.email = "http://groups.google.com/group/rfmcommunity" - gem.homepage = "http://sixfriedrice.com/wp/products/rfm/" - gem.authors = ["Geoff Coffey", "Mufaddal Khumri", "Atsushi Matsuo", "Larry Sprock"] - gem.files = FileList['lib/**/*'] - gem.add_dependency('nokogiri') - gem.rdoc_options = [ "--line-numbers", "--main", "README.rdoc" ] +task :default => :spec + +#require 'spec/rake/spectask' +#require 'rspec' +require 'rspec/core/rake_task' + +# Manual +# desc "Manually run rspec 2 - works but ugly" +# task :spec do +# puts exec("rspec -O spec/spec.opts") #RUBYOPTS=W0 # silence ruby warnings. +# end + +RSpec::Core::RakeTask.new(:spec) do |task| + # Optionally pass env var SPEC_OPTS='--whatever' to pass opts to rspec thru rake. + task.rspec_opts = '-O spec/spec.opts' +end + +desc "run specs with all parser backends" +task :spec_multi do + require 'benchmark' + require 'yaml' + Benchmark.bm do |b| + [:ox, :libxml, :nokogiri, :rexml].each do |parser| + ENV['parser'] = parser.to_s + b.report("RSPEC with #{parser.to_s.upcase}\n") do + begin + Rake::Task["spec"].execute + rescue Exception + puts "ERROR in rspec with #{parser.to_s.upcase}: #{$!}" + end + end + end end - Jeweler::GemcutterTasks.new -rescue LoadError - puts "Jeweler (or a dependency) not available. Install it with: sudo gem install jeweler" end -require 'spec/rake/spectask' -Spec::Rake::SpecTask.new(:spec) do |spec| - spec.libs << 'lib' << 'spec' - spec.spec_files = FileList['spec/**/*_spec.rb'] +require 'rdoc/task' +Rake::RDocTask.new do |rdoc| + version = Rfm::VERSION + rdoc.main = 'README.md' + rdoc.rdoc_dir = 'rdoc' + rdoc.title = "Rfm #{version}" + rdoc.rdoc_files.include('lib/**/*.rb', 'README.md', 'CHANGELOG.md', 'VERSION', 'LICENSE') end -Spec::Rake::SpecTask.new(:rcov) do |spec| - spec.libs << 'lib' << 'spec' - spec.pattern = 'spec/**/*_spec.rb' - spec.rcov = true +require 'yard' +require 'rdoc' +YARD::Rake::YardocTask.new do |t| + # See http://rubydoc.info/docs/yard/file/docs/GettingStarted.md + # See 'yardoc --help' + t.files = ['lib/**/*.rb', 'README.md', 'LICENSE', 'VERSION', 'CHANGELOG.md'] # optional + t.options = ['-oydoc', '--no-cache', '-mrdoc', '--no-private'] # optional end -task :default => :spec +desc "Print the version of Rfm" +task :version do + puts Rfm::VERSION +end -require 'rake/rdoctask' -Rake::RDocTask.new do |rdoc| - if File.exist?('VERSION.yml') - config = YAML.load(File.read('VERSION.yml')) - version = "#{config[:major]}.#{config[:minor]}.#{config[:patch]}" - else - version = "" +desc "Print info about Rfm" +task :info do + puts Rfm.info +end + +desc "Benchmark loading & parsing XML data into Rfm classes" +task :benchmark do + require 'benchmark' + require 'yaml' + #load "spec/data/sax_models.rb" + @records = File.read 'spec/data/resultset_large.xml' + #@template = 'lib/rfm/utilities/sax/fmresultset.yml' + @template = 'fmresultset.yml' + @base_class = Rfm::Resultset + @layout = 'spec/data/layout.xml' + Benchmark.bm do |b| + [:rexml, :nokogiri, :libxml, :ox].each do |backend| + b.report("#{backend}\n") do + 5.times do + Rfm::SaxParser.parse(@records, @template, @base_class.new, backend) + end + end + end end +end - rdoc.rdoc_dir = 'rdoc' - rdoc.title = "rfm #{version}" - rdoc.rdoc_files.include('README*') - rdoc.rdoc_files.include('lib/**/*.rb') +desc "Profile the sax parser" +task :profile_sax do + # This turns on tail-call-optimization. Was needed in past for ruby 1.9, maybe not now. + # See http://ephoz.posterous.com/ruby-and-recursion-whats-up-in-19 + # if RUBY_VERSION[/1.9/] + # RubyVM::InstructionSequence.compile_option = { + # :tailcall_optimization => true, + # :trace_instruction => false + # } + # end + require 'ruby-prof' + # Profile the code + @data = 'spec/data/resultset_large.xml' + min = ENV['min'] + min_percent= min ? min.to_i : 1 + result = RubyProf.profile do + # The parser will choose the best available backend. + @rr = Rfm::SaxParser.parse(@data, 'fmresultset.yml', Rfm::Resultset.new).result + #Rfm::SaxParser.parse(@data, 'lib/rfm/utilities/sax/fmresultset.yml', Rfm::Resultset.new) + end + + # Print a flat profile to text + flat_printer = RubyProf::FlatPrinter.new(result) + flat_printer.print(STDOUT, {:min_percent=>0}) + # Print a graph profile to text. See https://github.com/ruby-prof/ruby-prof/blob/master/examples/graph.txt + graph_printer = RubyProf::GraphPrinter.new(result) + graph_printer.print(STDOUT, {:min_percent=>min_percent}) + graph_html_printer = RubyProf::GraphHtmlPrinter.new(result) + File.open('ruby-prof-graph.html', 'w') do |f| + graph_html_printer.print(f, {:min_percent=>min_percent}) + end + # Print call_tree to html + tree_printer = RubyProf::CallStackPrinter.new(result) + File.open('ruby-prof-stack.html', 'w') do |f| + tree_printer.print(f, {:min_percent=>min_percent}) + end + + puts @rr.class + puts @rr.size + puts @rr[5].to_yaml end +desc "Run test data thru the sax parser" +task :sample do + @records = 'spec/data/resultset_large.xml' + r= Rfm::SaxParser.parse(@records, 'lib/rfm/utilities/sax/fmresultset.yml', Rfm::Resultset.new).result + puts r.to_yaml + puts r.field_meta.to_yaml +end + +desc "pre-commit, build gem, tag with version, push to git, push to rubygems.org" +task :release do + gem_name = 'ginjo-rfm' + shell = <<-EEOOFF + current_branch=`git rev-parse --abbrev-ref HEAD` + if [ $current_branch != 'master' ]; then + echo "Aborting: You are not on the master branch." + exit 1 + fi + echo "--- Pre-committing ---" + git add .; git commit -m'Releasing version #{Rfm::VERSION}' + echo "--- Building gem ---" && + mkdir -p pkg && + output=`gem build #{gem_name}.gemspec` && + gemfile=`echo "$output" | awk '{ field = $NF }; END{ print field }'` && + echo $gemfile && + mv -f $gemfile pkg/ && + echo "--- Tagging with git ---" && + git tag -m'Releasing version #{Rfm::VERSION}' v#{Rfm::VERSION} && + echo "--- Pushing to git origin ---" && + git push origin master && + git push origin --tags && + echo "--- Pushing to rubygems.org ---" && + gem push pkg/$gemfile + EEOOFF + puts "----- Shell script to release gem -----" + puts shell + puts "----- End shell script -----" + print exec(shell) +end + + diff --git a/VERSION b/VERSION deleted file mode 100644 index 347f5833..00000000 --- a/VERSION +++ /dev/null @@ -1 +0,0 @@ -1.4.1 diff --git a/VERSION b/VERSION new file mode 120000 index 00000000..97c309f3 --- /dev/null +++ b/VERSION @@ -0,0 +1 @@ +./lib/rfm/VERSION \ No newline at end of file diff --git a/ginjo-rfm.gemspec b/ginjo-rfm.gemspec new file mode 100644 index 00000000..838abd9c --- /dev/null +++ b/ginjo-rfm.gemspec @@ -0,0 +1,49 @@ +#!/usr/bin/env ruby +# -*- encoding: utf-8 -*- +# This gemspec has been crafted by hand - do not overwrite with Jeweler! +# See http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/ +# See http://yehudakatz.com/2010/04/02/using-gemspecs-as-intended/ +# for more information on bundler and gems. + +require 'date' + +Gem::Specification.new do |s| + s.name = "ginjo-rfm" + s.summary = "Ruby Filemaker adapter" + s.version = File.read('./lib/rfm/VERSION') #Rfm::VERSION + + s.required_rubygems_version = Gem::Requirement.new("> 1.3.1") if s.respond_to? :required_rubygems_version= + #s.authors = ["Geoff Coffey", "Mufaddal Khumri", "Atsushi Matsuo", "Larry Sprock", "Bill Richardson"] + s.authors = ["Bill Richardson", "Geoff Coffey", "Mufaddal Khumri", "Atsushi Matsuo", "Larry Sprock"] + s.date = Date.today.to_s + s.description = "Rfm is a standalone database adapter for Filemaker server. Ginjo-rfm features multiple xml parser support, ActiveModel integration, field mapping, compound queries, logging, scoping, and a configuration framework." + s.email = "http://groups.google.com/group/rfmcommunity" + s.homepage = "https://github.com/ginjo/rfm" + + s.require_paths = ["lib"] + s.files = Dir['lib/**/*.rb', 'lib/**/sax/*', 'lib/**/VERSION', '.yardopts'] + + s.rdoc_options = ["--line-numbers", "--main", "README.md"] + s.extra_rdoc_files = [ + "LICENSE", + "README.md", + "CHANGELOG.md", + "lib/rfm/VERSION" + ] + + # s.add_runtime_dependency('activesupport', '>= 2.3.5') + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 2"]) + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 0"]) unless RUBY_PLATFORM == 'java' + s.add_development_dependency(%q, [">= 0"]) + s.add_development_dependency(%q, [">= 0"]) + +end # Gem::Specification.new + diff --git a/lardawge-rfm.gemspec b/lardawge-rfm.gemspec deleted file mode 100644 index 97fab669..00000000 --- a/lardawge-rfm.gemspec +++ /dev/null @@ -1,55 +0,0 @@ -# Generated by jeweler -# DO NOT EDIT THIS FILE DIRECTLY -# Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command -# -*- encoding: utf-8 -*- - -Gem::Specification.new do |s| - s.name = %q{lardawge-rfm} - s.version = "1.4.1" - - s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version= - s.authors = ["Geoff Coffey", "Mufaddal Khumri", "Atsushi Matsuo", "Larry Sprock"] - s.date = %q{2010-06-28} - s.description = %q{Rfm brings your FileMaker data to Ruby. Now your Ruby scripts and Rails applications can talk directly to your FileMaker server.} - s.email = %q{http://groups.google.com/group/rfmcommunity} - s.extra_rdoc_files = [ - "LICENSE", - "README.rdoc" - ] - s.files = [ - "lib/rfm.rb", - "lib/rfm/database.rb", - "lib/rfm/error.rb", - "lib/rfm/layout.rb", - "lib/rfm/metadata/field.rb", - "lib/rfm/metadata/script.rb", - "lib/rfm/record.rb", - "lib/rfm/resultset.rb", - "lib/rfm/server.rb", - "lib/rfm/utilities/case_insensitive_hash.rb", - "lib/rfm/utilities/factory.rb" - ] - s.homepage = %q{http://sixfriedrice.com/wp/products/rfm/} - s.rdoc_options = ["--line-numbers", "--main", "README.rdoc"] - s.require_paths = ["lib"] - s.rubygems_version = %q{1.3.7} - s.summary = %q{Ruby to Filemaker adapter} - s.test_files = [ - "spec/rfm/error_spec.rb", - "spec/spec_helper.rb" - ] - - if s.respond_to? :specification_version then - current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION - s.specification_version = 3 - - if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then - s.add_runtime_dependency(%q, [">= 0"]) - else - s.add_dependency(%q, [">= 0"]) - end - else - s.add_dependency(%q, [">= 0"]) - end -end - diff --git a/lib/rfm.rb b/lib/rfm.rb index 4f440ce9..85aef4ba 100644 --- a/lib/rfm.rb +++ b/lib/rfm.rb @@ -1,19 +1,106 @@ -path = File.expand_path(File.dirname(__FILE__)) -$:.unshift(path) unless $:.include?(path) +module Rfm + PATH = File.expand_path(File.dirname(__FILE__)) + $LOAD_PATH.unshift(PATH) unless $LOAD_PATH.include?(PATH) +end + +#require 'thread' # some versions of ActiveSupport will raise error about Mutex unless 'thread' is loaded. -require path + '/rfm/utilities/case_insensitive_hash' -require path + '/rfm/utilities/factory' +require 'logger' +require 'rfm/utilities/core_ext' +require 'rfm/utilities/case_insensitive_hash' module Rfm - + class CommunicationError < StandardError; end class ParameterError < StandardError; end class AuthenticationError < StandardError; end - autoload :Error, 'rfm/error' - autoload :Server, 'rfm/server' - autoload :Database, 'rfm/database' - autoload :Layout, 'rfm/layout' - autoload :Resultset, 'rfm/resultset' - -end \ No newline at end of file + autoload :Error, 'rfm/error' + autoload :Server, 'rfm/server' + autoload :Database, 'rfm/database' + autoload :Layout, 'rfm/layout' + autoload :Resultset, 'rfm/resultset' + autoload :Record, 'rfm/record' + autoload :Base, 'rfm/base' + autoload :SaxParser, 'rfm/utilities/sax_parser' + autoload :Config, 'rfm/utilities/config' + autoload :Factory, 'rfm/utilities/factory' + autoload :CompoundQuery,'rfm/utilities/compound_query' + autoload :VERSION, 'rfm/version' + autoload :Connection, 'rfm/utilities/connection.rb' + autoload :Scope, 'rfm/utilities/scope.rb' + + module Metadata + autoload :Script, 'rfm/metadata/script' + autoload :Field, 'rfm/metadata/field' + autoload :FieldControl, 'rfm/metadata/field_control' + autoload :ValueListItem, 'rfm/metadata/value_list_item' + autoload :Datum, 'rfm/metadata/datum' + autoload :ResultsetMeta, 'rfm/metadata/resultset_meta' + autoload :LayoutMeta, 'rfm/metadata/layout_meta' + end + + def info + rslt = <<-EEOOFF + Gem name: ginjo-rfm + Version: #{VERSION} + ActiveModel loadable? #{begin; Gem::Specification::find_all_by_name('activemodel')[0].version.to_s; rescue Exception; false; end} + ActiveModel loaded? #{defined?(ActiveModel) ? 'true' : 'false'} + XML default parser: #{SaxParser::Handler.get_backend} + Ruby: #{RUBY_VERSION} + EEOOFF + rslt.gsub!(/^[ \t]*/, '') + rslt + rescue + "Could not retrieve info: #{$!}" + end + + def info_short + "Using ginjo-rfm version #{::Rfm::VERSION} with #{SaxParser::Handler.get_backend}" + end + + def_delegators 'Rfm::Factory', :servers, :server, :db, :database, :layout + def_delegators 'Rfm::Config', :config, :get_config, :config_clear + def_delegators 'Rfm::Resultset', :load_data + + def models(*args) + Rfm::Factory.models(*args) + end + + def modelize(*args) + Rfm::Factory.modelize(*args) + end + + def logger + @@logger ||= get_config[:logger] || Logger.new(STDOUT).tap {|l| l.formatter = proc {|severity, datetime, progname, msg| "#{datetime}: Rfm-#{severity} #{msg}\n"}} + end + + alias_method :log, :logger + + def logger=(obj) + @@logger = obj + end + + # DEFAULT_CLASS = CaseInsensitiveHash + # TEMPLATE_PREFIX = File.join(File.dirname(__FILE__), 'rfm/utilities/sax/') + # TEMPLATES = { + # :fmpxmllayout => 'fmpxmllayout.yml', + # :fmresultset => 'fmresultset.yml', + # :fmpxmlresult => 'fmpxmlresult.yml', + # :none => nil + # } + + PARSER_DEFAULTS = { + :default_class => CaseInsensitiveHash, + :template_prefix => File.join(File.dirname(__FILE__), 'rfm/utilities/sax/'), + :templates => { + :fmpxmllayout => 'fmpxmllayout.yml', + :fmresultset => 'fmresultset.yml', + :fmpxmlresult => 'fmpxmlresult.yml', + :none => nil + } + } + + extend self + +end # Rfm diff --git a/lib/rfm/.gitattributes b/lib/rfm/.gitattributes new file mode 100644 index 00000000..e8e1868c --- /dev/null +++ b/lib/rfm/.gitattributes @@ -0,0 +1 @@ +VERSION merge=ours \ No newline at end of file diff --git a/lib/rfm/VERSION b/lib/rfm/VERSION new file mode 100644 index 00000000..d003324b --- /dev/null +++ b/lib/rfm/VERSION @@ -0,0 +1 @@ +3.0.12 \ No newline at end of file diff --git a/lib/rfm/base.rb b/lib/rfm/base.rb new file mode 100644 index 00000000..43a1896f --- /dev/null +++ b/lib/rfm/base.rb @@ -0,0 +1,310 @@ +module Rfm + + # Adds ability to create Rfm::Base model classes that behave similar to ActiveRecord::Base models. + # If you set your Rfm.config (or RFM_CONFIG) with your host, database, account, password, and + # any other server/database options, you can provide your models with nothing more than a layout. + # + # class Person < Rfm::Base + # config :layout => 'mylayout' + # end + # + # And similar to ActiveRecord, you can define callbacks, validations, attributes, and methods on your model. + # (if you have ActiveModel loaded). + # + # class Account < Rfm::Base + # config :layout=>'account_xml' + # before_create :encrypt_password + # validates :email, :presence => true + # validates :username, :presence => true + # attr_accessor :password + # end + # + # Then in your project, you can use these models just like ActiveRecord models. + # The query syntax and options are still Rfm under the hood. Treat your model + # classes like Rfm::Layout objects, with a few enhancements. + # + # @account = Account.new :username => 'bill', :password => 'pass' + # @account.email = 'my@email.com' + # @account.save! + # + # @person = Person.find({:name => 'mike'}, :max_records => 50)[0] + # @person.update_attributes(:name => 'Michael', :title => "Senior Partner") + # @person.save + + class Base < Rfm::Record #Hash + extend Config + config :parent => 'Rfm::Config' + + begin + require 'active_model' + include ActiveModel::Validations + include ActiveModel::Serialization + extend ActiveModel::Callbacks + include ActiveModel::Validations::Callbacks + define_model_callbacks(:create, :update, :destroy) + rescue LoadError, StandardError + def run_callbacks(*args) + yield + end + end + + def to_partial_path(object = self) #@object) + return 'some/partial/path' + ##### DISABLED HERE - ActiveModel Lint only needs a string ##### + ##### TODO: implement to_partial_path to return meaningful string. + # @partial_names[object.class.name] ||= begin + # object = object.to_model if object.respond_to?(:to_model) + + # object.class.model_name.partial_path.dup.tap do |partial| + # path = @view.controller_path + # partial.insert(0, "#{File.dirname(path)}/") if partial.include?(?/) && path.include?(?/) + # end + # end + end + + class << self + + # Access layout functions from base model + def_delegators :layout, :db, :server, :field_controls, :field_names, :value_lists, :total_count, + :query, :all, :delete, :portal_meta, :portal_names, :database, :table, :count, :ignore_bad_data + + def inherited(model) + (Rfm::Factory.models << model).uniq unless Rfm::Factory.models.include? model + model.config :parent=>'Rfm::Base' + end + + def config(*args) + super(*args){|options| @config.merge!(:layout=>options[:strings][0]) if options[:strings] && options[:strings][0]} + end + + # Access/create the layout object associated with this model + def layout + return @layout if @layout + # cnf = get_config + # raise "Could not get :layout from get_config in Base.layout method" unless cnf[:layout] #return unless cnf[:layout] + # @layout = Rfm::Factory.layout(cnf).sublayout + name = get_config[:layout] || 'test' # The 'test' was added to help active-model-lint tests pass. + @layout = Rfm::Factory.layout(name, self) #.sublayout + + # Added by wbr to give config hierarchy: layout -> model -> sublayout + #config :parent=>'parent_layout' + #config :parent=>'Rfm::Config' + #@layout.config model + #@layout.config :parent=>self + + @layout.model = self + @layout + end + + # # Access the parent layout of this model + # def parent_layout + # layout #.parent_layout + # end + + # Just like Layout#find, but searching by record_id will return a record, not a resultset. + def find(find_criteria, options={}) + #puts "base.find-#{layout.object_id}" + r = layout.find(find_criteria, options) + if ![Hash,Array].include?(find_criteria.class) and r.size == 1 + r[0] + else + r + end + rescue Rfm::Error::RecordMissingError + nil + end + + # Layout#any, returns single record, not resultset + def any(*args) + layout.any(*args)[0] + end + + # New record, save, (with callbacks & validations if ActiveModel is loaded) + def create(*args) + new(*args).send :create + end + + # Using this method will skip callbacks. Use instance method +#update+ instead + def edit(*args) + layout.edit(*args)[0] + end + + end # class << self + + + # Is this a newly created record, not saved yet? + def new_record? + return true if (self.record_id.nil? || self.record_id.empty?) + end + + # Reload record from database + # TODO: handle error when record has been deleted + # TODO: Move this to Rfm::Record. + def reload(force=false) + if (@mods.empty? or force) and record_id + @mods.clear + self.replace_with_fresh_data layout.find(self.record_id)[0] #self.class.find(self.record_id) + end + end + + # Mass update of record attributes, without saving. + def update_attributes(new_attr) + new_attr.each do |k,v| + k = k.to_s.downcase + if key?(k) || (layout.field_keys.include?(k.split('.')[0]) rescue nil) + @mods[k] = v + self[k] = v + else + instance_variable_set("@#{k}", v) + end + end + end + # # Mass update of record attributes, without saving. + # # TODO: return error or nil if input hash contains no recognizable keys. + # def update_attributes(new_attr) + # # creates new special hash + # input_hash = Rfm::CaseInsensitiveHash.new + # # populate new hash with input, coercing keys to strings + # #new_attr.each{|k,v| input_hash.merge! k.to_s=>v} + # new_attr.each{|k,v| input_hash[k.to_s] = v} + # # loop thru each layout field, adding data to @mods + # self.class.field_controls.keys.each do |field| + # field_name = field.to_s + # if input_hash.has_key?(field_name) + # #@mods.merge! field_name=>(input_hash[field_name] || '') + # @mods[field_name] = (input_hash[field_name] || '') + # end + # end + # # loop thru each input key-value, + # # creating new attribute if key doesn't exist in model. + # input_hash.each do |k,v| + # if !self.class.field_controls.keys.include?(k) and self.respond_to?(k) + # self.instance_variable_set("@#{k}", v) + # end + # end + # self.merge!(@mods) unless @mods == {} + # #self.merge!(@mods) unless @mods == Rfm::CaseInsensitiveHash.new + # end + + # Mass update of record attributes, with saving. + def update_attributes!(new_attr) + self.update_attributes(new_attr) + self.save! + end + + # Save record modifications to database (with callbacks & validations). If record cannot be saved will raise error. + def save! + #return unless @mods.size > 0 + raise "Record Invalid" unless valid? rescue nil + if @record_id + self.update + else + self.create + end + rescue + (self.errors[:base] rescue []) << $! + raise $! + end + + # Same as save!, but will not raise error. + def save + save! + # rescue + # (self.errors[:base] rescue []) << $! + # return nil + rescue + nil + end + + # Just like Layout#save_if_not_modified, but with callbacks & validations. + def save_if_not_modified + update(@mod_id) if @mods.size > 0 + end + + # Delete record from database, with callbacks & validations. + def destroy + return unless record_id + run_callbacks :destroy do + self.class.delete(record_id) + @destroyed = true + @mods.clear + end + self.freeze + #self + end + + def destroyed? + @destroyed + end + + # For ActiveModel compatibility + def to_model + self + end + + def persisted? + record_id ? true : false + end + + def to_key + record_id ? [record_id] : nil + end + + def to_param + record_id + end + + + protected # Base + + def self.create_from_new(*args) + layout.create(*args)[0] + end + + # shunt for callbacks when not using ActiveModel + def callback_deadend (*args) + yield #(*args) + end + + def create + raise "Record not valid" if (defined?(ActiveModel::Validations) && !valid?) + run_callbacks :create do + return unless @mods.size > 0 + # merge_rfm_result self.class.create_from_new(@mods) + replace_with_fresh_data self.class.create_from_new(@mods) + end + self + end + + def update(mod_id=nil) + raise "Record not valid" if (defined?(ActiveModel::Validations) && !valid?) + return false unless record_id + run_callbacks :update do + return unless @mods.size > 0 + unless mod_id + # regular save + # merge_rfm_result self.class.send :edit, record_id, @mods + replace_with_fresh_data self.class.send :edit, record_id, @mods + else + # save_if_not_modified + # merge_rfm_result self.class.send :edit, record_id, @mods, :modification_id=>mod_id + replace_with_fresh_data self.class.send :edit, record_id, @mods, :modification_id=>mod_id + end + end + self + end + + # Deprecated in favor of Record#replace_with_fresh_data + def merge_rfm_result(result_record) + return unless @mods.size > 0 + @record_id ||= result_record.record_id + self.merge! result_record + @mods.clear + self || {} + #self || Rfm::CaseInsensitiveHash.new + end + + end # Base + +end # Rfm + diff --git a/lib/rfm/database.rb b/lib/rfm/database.rb index d8e52ece..07d7560e 100644 --- a/lib/rfm/database.rb +++ b/lib/rfm/database.rb @@ -58,24 +58,61 @@ module Rfm # * *name* is the name of this database # * *state* is a hash of all server options used to initialize this server class Database - + include Config + # Initialize a database object. You never really need to do this. Instead, just do this: # # myServer = Rfm::Server.new(...) # myDatabase = myServer["Customers"] # # This sample code gets a database object representing the Customers database on the FileMaker server. - def initialize(name, server) - @name = name - @server = server - @account_name = server.state[:account_name] or "" - @password = server.state[:password] or "" - @layout = Rfm::Factory::LayoutFactory.new(server, self) - @script = Rfm::Factory::ScriptFactory.new(server, self) + def initialize(*args) #name, server_obj, acnt=nil, pass=nil + config(*args) + raise Rfm::Error::RfmError.new(0, "New instance of Rfm::Database has no name. Attempted name '#{state[:database]}'.") if state[:database].to_s == '' + @layouts = Rfm::Factory::LayoutFactory.new(server, self) + @scripts = Rfm::Factory::ScriptFactory.new(server, self) + end + + meta_attr_accessor :server + attr_reader :layouts, :scripts + # Not sure if these writers are ever used + #attr_writer :account_name, :password + # Legacy methods + alias_method :layout, :layouts + alias_method :script, :scripts + + def name + state[:database].to_s + end + + def account_name + state[:account_name] + end + + def account_name=(x) + config :account_name=>x + end + + def password + state[:password] + end + + def password=(x) + config :password=>x + end + + def config(*args) + super(:capture_strings_with=>[:database, :account_name, :password]) + super(*args) do |params| + (self.server = params[:objects][0]) if params && params[:objects] && params[:objects][0] && params[:objects][0].is_a?(Rfm::Server) + end end - - attr_reader :server, :name, :account_name, :password, :layout, :script - attr_writer :account_name, :password + + alias_method :server_orig, :server + def server + server_orig || (self.server = Rfm::Server.new(state[:host], state[:account_name], state[:password], self)) + end + # Access the Layout object representing a layout in this database. For example: # @@ -88,9 +125,10 @@ def initialize(name, server) # returned is created on the fly and assumed to refer to a valid layout, but you will # get no error at this point if the layout you specify doesn't exist. Instead, you'll # receive an error when you actually try to perform some action it. - def [](layout_name) - self.layout[layout_name] - end + # def [](layout_name) + # self.layout[layout_name] + # end + def_delegators :layouts, :[], :modelize, :models # modelize & models acquired from Rfm::Base end -end \ No newline at end of file +end diff --git a/lib/rfm/error.rb b/lib/rfm/error.rb index 7e798463..bf132e2b 100644 --- a/lib/rfm/error.rb +++ b/lib/rfm/error.rb @@ -1,38 +1,38 @@ module Rfm - + # Error is the base for the error hierarchy representing errors returned by Filemaker. - # + # # One could raise a FileMakerError by doing: # raise Rfm::Error.getError(102) # # It also takes an optional argument to give a more discriptive error message: # err = Rfm::Error.getError(102, 'add description with more detail here') - # + # # The above code would return a FieldMissing instance. Your could use this instance to raise that appropriate # exception: - # - # raise err - # + # + # raise err + # # You could access the specific error code by accessing: - # + # # err.code module Error - + class RfmError < StandardError #:nodoc: attr_reader :code - + def initialize(code, message=nil) @code = code super(message) end end - class UnknownError < RfmError + class UnknownError < RfmError end - + class SystemError < RfmError end - + class MissingError < RfmError end @@ -42,10 +42,10 @@ class RecordMissingError < MissingError #:nodoc: class FieldMissingError < MissingError #:nodoc: end - class ScriptMissingError < MissingError #:nodoc: + class ScriptMissingError < MissingError #:nodoc: end - class LayoutMissingError < MissingError #:nodoc: + class LayoutMissingError < MissingError #:nodoc: end class TableMissingError < MissingError #:nodoc: @@ -82,7 +82,7 @@ class NoRecordsFoundError < GeneralError #:nodoc: end class ValidationError < RfmError #:nodoc: - end + end class DateValidationError < ValidationError #:nodoc: end @@ -90,7 +90,7 @@ class DateValidationError < ValidationError #:nodoc: class TimeValidationError < ValidationError #:nodoc: end - class NumberValidationError < ValidationError #:nodoc: + class NumberValidationError < ValidationError #:nodoc: end class RangeValidationError < ValidationError #:nodoc: @@ -115,11 +115,11 @@ class MaximumCharactersValidationError < ValidationError #:nodoc: end class FileError < RfmError #:nodoc: - end + end class UnableToOpenFileError < FileError #:nodoc: end - + extend self # This method returns the appropriate FileMaker object depending on the error code passed to it. It # also accepts an optional message. @@ -129,15 +129,15 @@ def getError(code, message=nil) error = klass.new(code, message) error end - + def build_message(klass, code, message=nil) #:nodoc: msg = ": #{message}" msg << " " unless message.nil? msg << "(FileMaker Error ##{code})" - + "#{klass.to_s.gsub(/Rfm::Error::/, '')} occurred#{msg}" end - + def find_by_code(code) #:nodoc: case code when 0..99 then SystemError @@ -159,8 +159,8 @@ def find_by_code(code) #:nodoc: elsif code == 306; RecordModIdDoesNotMatchError else; ConcurrencyError; end when 400..499 - if code == 401; NoRecordsFoundError - else; GeneralError; end + if code == 401; NoRecordsFoundError + else; GeneralError; end when 500..599 if code == 500; DateValidationError elsif code == 501; TimeValidationError @@ -182,5 +182,5 @@ def find_by_code(code) #:nodoc: end end end - -end \ No newline at end of file + +end diff --git a/lib/rfm/layout.rb b/lib/rfm/layout.rb index 1a639326..7ccf5c25 100644 --- a/lib/rfm/layout.rb +++ b/lib/rfm/layout.rb @@ -1,3 +1,5 @@ +require 'delegate' + module Rfm # The Layout object represents a single FileMaker Pro layout. You use it to interact with # records in FileMaker. *All* access to FileMaker data is done through a layout, and this @@ -117,9 +119,47 @@ module Rfm # * +value_lists+ is a hash of arrays. The keys are value list names, and the values in the hash # are arrays containing the actual value list items. +value_lists+ will include every value # list that is attached to any field on the layout - + class Layout - + include Config + + meta_attr_accessor :db + attr_reader :field_mapping + attr_writer :resultset_meta + def_delegator :db, :server + #alias_method :database, :db # This fails if db object hasn't been set yet with meta_attr_accessor + + def database + db + end + + attr_accessor :model #, :parent_layout, :subs + def_delegators :meta, :field_controls, :value_lists + def_delegators :resultset_meta, :date_format, :time_format, :timestamp_format, :field_meta, :portal_meta, :table + + # Methods that must be kept after rewrite!!! + # + # field_mapping + # db (database) + # name + # resultset_meta + # date_format + # time_format + # timestamp_format + # field_meta + # field_controls + # field_names + # field_names_no_load + # value_lists + # count + # total_count + # portal_meta + # portal_meta_no_load + # portal_names + # table + # table_no_load + # server + # Initialize a layout object. You never really need to do this. Instead, just do this: # # myServer = Rfm::Server.new(...) @@ -133,23 +173,39 @@ class Layout # # myServer = Rfm::Server.new(...) # myLayout = myServer["Customers"]["Details"] - def initialize(name, db) - @name = name - @db = db + + def initialize(*args) #name, db_obj + # self.subs ||= [] + config(*args) + raise Rfm::Error::RfmError.new(0, "New instance of Rfm::Layout has no name. Attempted name '#{state[:layout]}'.") if state[:layout].to_s == '' + @loaded = false + @meta = Metadata::LayoutMeta.new(self) + self + end + + def config(*args) + super(:capture_strings_with=>[:layout]) + super(*args) do |params| + (self.name = params[:strings][0]) if params && params[:strings] && params[:strings].any? + (self.db = params[:objects][0]) if params && params[:objects] && params[:objects][0] && params[:objects][0].is_a?(Rfm::Database) + end end - - attr_reader :name, :db - + + alias_method :db_orig, :db + def db + db_orig || (self.db = Rfm::Database.new(state[:database], state[:account_name], state[:password], self)) + end + # Returns a ResultSet object containing _every record_ in the table associated with this layout. def all(options = {}) get_records('-findall', {}, options) end - + # Returns a ResultSet containing a single random record from the table associated with this layout. def any(options = {}) get_records('-findany', {}, options) end - + # Finds a record. Typically you will pass in a hash of field names and values. For example: # # myLayout.find({"First Name" => "Bill"}) @@ -157,16 +213,34 @@ def any(options = {}) # Values in the hash work just like value in FileMaker's Find mode. You can use any special # symbols (+==+, +...+, +>+, etc...). # - # If you pass anything other than a hash as the first parameter, it is converted to a string and + # Create a Filemaker 'omit' request by including an :omit key with a value of true. + # + # myLayout.find :field1 => 'val1', :field2 => 'val2', :omit => true + # + # Create multiple Filemaker find requests by passing an array of hashes to the #find method. + # + # myLayout.find [{:field1 => 'bill', :field2 => 'admin'}, {:field3 => 'inactive', :omit => true}, ...] + # + # If the value of a field in a find request is an array of strings, the string values will be logically OR'd in the query. + # + # myLayout.find :fieldOne => ['bill','mike','bob'], :fieldTwo =>'staff' + # + # If you pass anything other than a hash or an array as the first parameter, it is converted to a string and # assumed to be FileMaker's internal id for a record (the recid). - def find(hash_or_recid, options = {}) - if hash_or_recid.kind_of? Hash - get_records('-find', hash_or_recid, options) - else - get_records('-find', {'-recid' => hash_or_recid.to_s}, options) - end + # + # myLayout.find 54321 + # + def find(find_criteria, options = {}) + #puts "layout.find-#{self.object_id}" + options.merge!({:field_mapping => field_mapping.invert}) if field_mapping + get_records(*Rfm::CompoundQuery.new(find_criteria, options)) end - + + # Access to raw -findquery command. + def query(query_hash, options = {}) + get_records('-findquery', query_hash, options) + end + # Updates the contents of the record whose internal +recid+ is specified. Send in a hash of new # data in the +values+ parameter. Returns a RecordSet containing the modified record. For example: # @@ -177,8 +251,9 @@ def find(hash_or_recid, options = {}) # first name to _Steve_. def edit(recid, values, options = {}) get_records('-edit', {'-recid' => recid}.merge(values), options) + #get_records('-edit', {'-recid' => recid}.merge(expand_repeats(values)), options) # attempt to set repeating fields. end - + # Creates a new record in the table associated with this layout. Pass field data as a hash in the # +values+ parameter. Returns the newly created record in a RecordSet. You can use the returned # record to, ie, discover the values in auto-enter fields (like serial numbers). @@ -193,7 +268,7 @@ def edit(recid, values, options = {}) def create(values, options = {}) get_records('-new', values, options) end - + # Deletes the record with the specified internal recid. Returns a ResultSet with the deleted record. # # For example: @@ -206,17 +281,189 @@ def delete(recid, options = {}) get_records('-delete', {'-recid' => recid}, options) return nil end - - private - + + # Retrieves metadata only, with an empty resultset. + def view(options = {}) + get_records('-view', {}, options) + end + + # Get the foundset_count only given criteria & options. + def count(find_criteria, options={}) + find(find_criteria, options.merge({:max_records => 0})).foundset_count + end + def get_records(action, extra_params = {}, options = {}) - include_portals = options[:include_portals] ? options.delete(:include_portals) : nil - xml_response = @db.server.connect(@db.account_name, @db.password, action, params.merge(extra_params), options).body - Rfm::Resultset.new(@db.server, xml_response, self, include_portals) + # TODO: See auto-grammar bypbass in connection.rb. + grammar_option = state(options)[:grammar] + options.merge!(:grammar=>grammar_option) if grammar_option + template = options.delete :template + + # # TODO: Remove this code it is no longer used. + # #include_portals = options[:include_portals] ? options.delete(:include_portals) : nil + # include_portals = !options[:ignore_portals] + + # Apply mapping from :field_mapping, to send correct params in URL. + prms = params.merge(extra_params) + map = field_mapping.invert + options.merge!({:field_mapping => map}) if map && !map.empty? + # TODO: Make this part handle string AND symbol keys. (isn't this already done?) + #map.each{|k,v| prms[k]=prms.delete(v) if prms[v]} + + #prms.dup.each_key{|k| prms[map[k.to_s]]=prms.delete(k) if map[k.to_s]} + prms.dup.each_key do |k| + new_key = map[k.to_s] || k + if prms[new_key].is_a? Array + prms[new_key].each_with_index do |v, i| + prms["#{new_key}(#{i+1})"]=v + end + prms.delete new_key + else + prms[new_key]=prms.delete(k) if new_key != k + end + #puts "PRMS: #{new_key} #{prms[new_key].class} #{prms[new_key]}" + end + + #c = Connection.new(action, prms, options, state.merge(:parent=>self)) + c = Connection.new(action, prms, options, self) + #rslt = c.parse(template || :fmresultset, Rfm::Resultset.new(self, self)) + rslt = c.parse(template, Rfm::Resultset.new(self, self)) + capture_resultset_meta(rslt) unless resultset_meta_valid? #(@resultset_meta && @resultset_meta.error != '401') + rslt end - + def params - {"-db" => @db.name, "-lay" => self.name} + {"-db" => state[:database], "-lay" => self.name} + end + + def name + state[:layout].to_s + end + + + ### Metadata from Layout ### + + def meta + @loaded ? @meta : load_layout + end + + def field_names + case + when @loaded + meta.field_names + when @resultset_meta + resultset_meta.field_names + else meta.field_names + end + end + + def field_names + # case + # when @loaded; meta.field_names + # when @resultset_meta; resultset_meta.field_names + # else meta.field_names + # end + meta.field_names + end + + def field_keys + # case + # when @loaded; @meta.field_keys + # when @resultset_meta; @resultset_meta.field_keys + # else meta.field_keys + # end + meta.field_keys + end + + + + ### Metadata from Resultset ### + + def resultset_meta + #@resultset_meta || view.meta + resultset_meta_valid? ? @resultset_meta : view.meta + end + + def resultset_meta_valid? + if @resultset_meta && @resultset_meta.error != '401' + true + end + end + + # Should always refresh + def total_count + view.total_count + end + + def capture_resultset_meta(resultset) + (@resultset_meta = resultset.clone.replace([])) #unless @resultset_meta + @resultset_meta = resultset.meta + end + + def portal_names + return 'UNDER-CONTSTRUCTION' + end + + + + + ### Utility ### + + def load_layout + #@loaded = true # This is first so parsing call to 'meta' wont cause infinite loop, + # but I changed parsing template to refer directly to inst var instead of accessor method. + connection = Connection.new('-view', {'-db' => state[:database], '-lay' => name}, {:grammar=>'FMPXMLLAYOUT'}, self) + begin + connection.parse(:fmpxmllayout, self) + @loaded = true + rescue + @meta.clear + raise $! + end + @meta + end + + def check_for_errors(code=@meta['error'].to_i, raise_401=state[:raise_401]) + #puts ["\nRESULTSET#check_for_errors", code, raise_401] + raise Rfm::Error.getError(code) if code != 0 && (code != 401 || raise_401) + end + + def field_mapping + @field_mapping ||= load_field_mapping(state[:field_mapping]) + end + + def load_field_mapping(mapping={}) + mapping = (mapping || {}).to_cih + def mapping.invert + super.to_cih + end + mapping + end + + # Creates new class with layout name. + def modelize + @model ||= ( + model_name = name.to_s.gsub(/\W|_/, ' ').title_case.gsub(/\s/,'') + #model_class = eval("::" + model_name + "= Class.new(Rfm::Base)") + model_class = Rfm.const_defined?(model_name) ? Rfm.const_get(model_name) : Rfm.const_set(model_name, Class.new(Rfm::Base)) + model_class.class_exec(self) do |layout_obj| + @layout = layout_obj + end + model_class.config :parent=>'@layout' + model_class + ) + # rescue StandardError, SyntaxError + # puts "Error in layout#modelize: #{$!}" + # nil + end + + def models + #subs.collect{|s| s.model} + [@model] end - end -end \ No newline at end of file + + + private :load_layout, :get_records, :params, :check_for_errors + + + end # Layout +end # Rfm diff --git a/lib/rfm/metadata/datum.rb b/lib/rfm/metadata/datum.rb new file mode 100644 index 00000000..8d67b5a9 --- /dev/null +++ b/lib/rfm/metadata/datum.rb @@ -0,0 +1,48 @@ +#require 'delegate' +module Rfm + module Metadata + + class Datum #< DelegateClass(Field) + + def get_mapped_name(name, resultset) + #puts ["\nDATUM#get_mapped_name", "name: #{name}", "mapping: #{resultset.layout.field_mapping.to_yaml}"] + (resultset && resultset.layout && resultset.layout.field_mapping[name]) || name + end + + # NOT sure what this method is for. Can't find a reference to it. + def main_callback(cursor) + resultset = cursor.top.object + name = get_mapped_name(@attributes['name'].to_s, resultset) + field = resultset.field_meta[name] + data = @attributes['data'] + cursor.parent.object[name.downcase] = field.coerce(data) + end + + def portal_field_element_close_callback(cursor) + resultset = cursor.top.object + table, name = @attributes['name'].to_s.split('::') + #puts ['DATUM_portal_field_element_close_callback_01', table, name].join(', ') + name = get_mapped_name(name, resultset) + field = resultset.portal_meta[table.downcase][name.downcase] + data = @attributes['data'] + #puts ['DATUM_portal_field_element_close_callback_02', "cursor.parent.object.class: #{cursor.parent.object.class}", "resultset.class: #{resultset.class}", "table: #{table}", "name: #{name}", "field: #{field}", "data: #{data}"] + #(puts resultset.portal_meta.to_yaml) unless field + cursor.parent.object[name.downcase] = field.coerce(data) + end + + # Should return value only. + def field_element_close_callback(cursor) + record = cursor.parent.object + resultset = cursor.top.object + + name = get_mapped_name(@attributes['name'].to_s, resultset) + field = resultset.field_meta[name] + data = @attributes['data'] #'data' + #puts ["\nDATUM", name, record.class, resultset.class, data] + #puts ["\nDATUM", self.to_yaml] + record[name] = field.coerce(data) + end + + end # Field + end # Metadata +end # Rfm diff --git a/lib/rfm/metadata/field.rb b/lib/rfm/metadata/field.rb index f917e47a..d6de1ce6 100644 --- a/lib/rfm/metadata/field.rb +++ b/lib/rfm/metadata/field.rb @@ -1,7 +1,10 @@ +require 'forwardable' + module Rfm module Metadata + # The Field object represents a single FileMaker field. It *does not hold the data* in the field. Instead, - # it serves as a source of metadata about the field. For example, if you're script is trying to be highly + # it serves as a source of metadata about the field. For example, if your script is trying to be highly # dynamic about its field access, it may need to determine the data type of a field at run time. Here's # how: # @@ -56,38 +59,78 @@ module Metadata # # The code above makes sure the control is always an array. Typically, though, you'll know up front # if the control is an array or not, and you can code accordingly. - class Field - + attr_reader :name, :result, :type, :max_repeats, :global - + meta_attr_accessor :resultset_meta + def_delegator :resultset_meta, :layout_object, :layout_object # Initializes a field object. You'll never need to do this. Instead, get your Field objects from - # ResultSet::fields - def initialize(field) - @name = field['name'] - @result = field['result'] - @type = field['type'] - @max_repeats = field['max-repeats'] - @global = field['global'] + # Resultset::field_meta + def initialize(attributes) + if attributes && attributes.size > 0 + _attach_as_instance_variables attributes + end + self end - + # Coerces the text value from an +fmresultset+ document into proper Ruby types based on the # type of the field. You'll never need to do this: Rfm does it automatically for you when you # access field data through the Record object. - def coerce(value, resultset) - return nil if value.empty? - case self.result + def coerce(value) + case + when (value.nil? or value.empty?) + return nil + when value.is_a?(Array) + return value.collect {|v| coerce(v)} + when value.is_a?(Hash) + return coerce(value.values[0]) + end + + case result when "text" then value - when "number" then BigDecimal.new(value) - when "date" then Date.strptime(value, resultset.date_format) - when "time" then DateTime.strptime("1/1/-4712 #{value}", "%m/%d/%Y #{resultset.time_format}") - when "timestamp" then DateTime.strptime(value, resultset.timestamp_format) - when "container" then URI.parse("#{resultset.server.scheme}://#{resultset.server.host_name}:#{resultset.server.port}#{value}") + when "number" then + # FileMaker uses local number format of the server. It stores input as text. + # So on european servers a number value of '1.200,50' is considered valid and interpreted as 1200.5 + # To support this, :decimal_separator can be configured and any other characters than digits and the separator are ignored + sep = layout_object.state[:decimal_separator] || '.' + BigDecimal.new(value.to_s.gsub(/[^-^\d#{Regexp.quote(sep)}]/,'').tr(sep,'.')) + when "date" then Date.strptime(value, resultset_meta.date_format) + when "time" then DateTime.strptime("1/1/-4712 #{value}", "%m/%d/%Y #{resultset_meta.time_format}") + when "timestamp" then DateTime.strptime(value, resultset_meta.timestamp_format) + when "container" then + #resultset_meta = resultset.instance_variable_get(:@meta) + if resultset_meta && resultset_meta['doctype'] && value.to_s[/\?/] + URI.parse(resultset_meta['doctype'].last.to_s).tap{|uri| uri.path, uri.query = value.split('?')} + else + value + end else nil end - + rescue + puts("ERROR in Field#coerce:", name, value, result, resultset_meta.timestamp_format, $!) + nil + end + + def get_mapped_name + #(resultset_meta && resultset_meta.layout && resultset_meta.layout.field_mapping[name]) || name + layout_object.field_mapping[name] || name + end + + def field_definition_element_close_callback(cursor) + #self.resultset = cursor.top.object + #resultset_meta = resultset.instance_variable_get(:@meta) + self.resultset_meta = cursor.top.object.instance_variable_get(:@meta) + #puts ["\nFIELD#field_definition_element_close_callback", resultset_meta] + resultset_meta.field_meta[get_mapped_name.to_s.downcase] = self + end + + def relatedset_field_definition_element_close_callback(cursor) + #self.resultset = cursor.top.object + self.resultset_meta = cursor.top.object.instance_variable_get(:@meta) + cursor.parent.object[get_mapped_name.split('::').last.to_s.downcase] = self + #puts ['FIELD_portal_callback', name, cursor.parent.object.object_id, cursor.parent.tag, cursor.parent.object[name.split('::').last.to_s.downcase]].join(', ') end - - end - end -end \ No newline at end of file + + end # Field + end # Metadata +end # Rfm diff --git a/lib/rfm/metadata/field_control.rb b/lib/rfm/metadata/field_control.rb new file mode 100644 index 00000000..a5a7752b --- /dev/null +++ b/lib/rfm/metadata/field_control.rb @@ -0,0 +1,84 @@ +module Rfm + module Metadata + + # The FieldControl object represents a field on a FileMaker layout. You can find out what field + # style the field uses, and the value list attached to it. + # + # =Attributes + # + # * *name* is the name of the field + # + # * *style* is any one of: + # * * :edit_box - a normal editable field + # * * :scrollable - an editable field with scroll bar + # * * :popup_menu - a pop-up menu + # * * :checkbox_set - a set of checkboxes + # * * :radio_button_set - a set of radio buttons + # * * :popup_list - a pop-up list + # * * :calendar - a pop-up calendar + # + # * *value_list_name* is the name of the attached value list, if any + # + # * *value_list* is an array of strings representing the value list items, or nil + # if this field has no attached value list + class FieldControl + attr_reader :name, :style, :value_list_name + meta_attr_accessor :layout_meta + + FIELD_CONTROL_STYLE_MAP = { + 'EDITTEXT' => :edit_box, + 'POPUPMENU' => :popup_menu, + 'CHECKBOX' => :checkbox_set, + 'RADIOBUTTONS' => :radio_button_set, + 'POPUPLIST' => :popup_list, + 'CALENDAR' => :calendar, + 'SCROLLTEXT' => :scrollable, + } + + # def initialize(_attributes, meta) + # puts ["\nFieldControl#initialize", "_attributes: #{_attributes}", "meta: #{meta.class}"] + # self.layout_meta = meta + # _attach_as_instance_variables(_attributes) if _attributes + # self + # end + + def initialize(meta) + #puts ["\nFieldControl#initialize", "meta: #{meta.class}"] + self.layout_meta = meta + self + end + + # # Handle manual attachment of STYLE element. + # def handle_style_element(attributes) + # _attach_as_instance_variables attributes, :key_translator=>method(:translate_value_list_key), :value_translator=>method(:translate_style_value) + # end + # + # def translate_style_value(key, val) + # #puts ["TRANSLATE_STYLE", raw].join(', ') + # { + # 'EDITTEXT' => :edit_box, + # 'POPUPMENU' => :popup_menu, + # 'CHECKBOX' => :checkbox_set, + # 'RADIOBUTTONS' => :radio_button_set, + # 'POPUPLIST' => :popup_list, + # 'CALENDAR' => :calendar, + # 'SCROLLTEXT' => :scrollable, + # }[val] || val + # end + + # def translate_value_list_key(raw) + # {'valuelist'=>'value_list_name'}[raw] || raw + # end + + def value_list + layout_meta.value_lists[value_list_name] + end + + def element_close_handler #(_cursor) + @type = FIELD_CONTROL_STYLE_MAP[@type] || @type + layout_meta.receive_field_control(self) + end + + end + end +end diff --git a/lib/rfm/metadata/layout_meta.rb b/lib/rfm/metadata/layout_meta.rb new file mode 100644 index 00000000..7df69a61 --- /dev/null +++ b/lib/rfm/metadata/layout_meta.rb @@ -0,0 +1,43 @@ +module Rfm + module Metadata + class LayoutMeta < CaseInsensitiveHash + + def initialize(layout) + @layout = layout + end + + def field_controls + self['field_controls'] ||= CaseInsensitiveHash.new + end + + def field_names + field_controls.values.collect{|v| v.name} + end + + def field_keys + field_controls.keys + end + + def value_lists + self['value_lists'] ||= CaseInsensitiveHash.new + end + + # def handle_new_field_control(attributes) + # name = attributes['name'] + # field_control = FieldControl.new(attributes, self) + # field_controls[get_mapped_name(name)] = field_control + # end + + def receive_field_control(fc) + #name = fc.name + field_controls[get_mapped_name(fc.name)] = fc + end + + # Should this be in FieldControl object? + def get_mapped_name(name) + (@layout.field_mapping[name]) || name + end + + end + end +end diff --git a/lib/rfm/metadata/resultset_meta.rb b/lib/rfm/metadata/resultset_meta.rb new file mode 100644 index 00000000..6b26f3bf --- /dev/null +++ b/lib/rfm/metadata/resultset_meta.rb @@ -0,0 +1,75 @@ +module Rfm + module Metadata + class ResultsetMeta < CaseInsensitiveHash + + def field_meta + self['field_meta'] ||= CaseInsensitiveHash.new + end + + def portal_meta + self['portal_meta'] ||= CaseInsensitiveHash.new + end + + def date_format + self['date_format'] + end + + def time_format + self['time_format'] + end + + def timestamp_format + self['timestamp_format'] + end + + def total_count + self['total_count'].to_i + end + + def foundset_count + self['count'].to_i + end + + def fetch_size + self['fetch_size'].to_i + end + + def table + self['table'] + end + + def error + self['error'] + end + + def field_names + field_meta ? field_meta.values.collect{|v| v.name} : [] + end + + def field_keys + field_meta ? field_meta.keys : [] + end + + def portal_names + portal_meta ? portal_meta.keys : [] + end + + # def handle_new_field(attributes) + # f = Field.new(attributes) + # # TODO: Re-enable these when you stop using the before_close callback. + # # name = attributes['name'] + # # self[name] = f + # end + + def layout_object + self['layout_object'] + end + + def attach_layout_object_from_cursor(cursor) + self['layout_object'] = cursor.top.object.layout + #puts ["\nRESULTSET_META#metadata_element_close_callback", self['layout_object']] + end + + end + end +end diff --git a/lib/rfm/metadata/script.rb b/lib/rfm/metadata/script.rb index 0807aa47..3b078a71 100644 --- a/lib/rfm/metadata/script.rb +++ b/lib/rfm/metadata/script.rb @@ -9,12 +9,14 @@ module Metadata # # If you want to _run_ a script, see the Layout object instead. class Script - def initialize(name, db) + def initialize(name, db_obj) @name = name - @db = db + self.db = db_obj end - + + meta_attr_accessor :db attr_reader :name - end - end -end \ No newline at end of file + end # Script + + end # Metadata +end # Rfm diff --git a/lib/rfm/metadata/value_list_item.rb b/lib/rfm/metadata/value_list_item.rb new file mode 100644 index 00000000..e655ec94 --- /dev/null +++ b/lib/rfm/metadata/value_list_item.rb @@ -0,0 +1,31 @@ +module Rfm + module Metadata + + # The ValueListItem object represents an item in a Filemaker value list. + # ValueListItem is subclassed from String, so you can use it just like + # a string. It does have three additional methods to help separate Filemaker *value* + # vs *display* items. + # + # Getting values vs display items: + # + # * *#value* the value list item value + # + # * *#display* is the value list item display. It could be the same + # as +value+, or it could be the "second field", if that option is checked in Filemaker + # + # * *#value_list_name* is the name of the parent value list, if any + class ValueListItem < String + # TODO: re-instate saving of value_list_name. + attr_reader :value, :display, :value_list_name + + # def initialize(value, display, value_list_name) + # @value_list_name = value_list_name + # @value = value.to_s + # @display = display.to_s + # self.replace @value + # end + + end # ValueListItem + + end # Metadata +end # Rfm diff --git a/lib/rfm/record.rb b/lib/rfm/record.rb index f8710bd5..bca61d86 100644 --- a/lib/rfm/record.rb +++ b/lib/rfm/record.rb @@ -1,5 +1,4 @@ module Rfm - # The Record object represents a single FileMaker record. You typically get them from ResultSet objects. # For example, you might use a Layout object to find some records: # @@ -58,6 +57,8 @@ module Rfm # In the above example, the Price field is a repeating field. The code puts the first repetition in a variable called # +val1+ and the second in a variable called +val2+. # + # It is not currently possible to create or edit a record's repeating fields beyond the first repitition, using Rfm. + # # =Accessing Portals # # If the ResultSet includes portals (because the layout it comes from has portals on it) you can access them @@ -69,7 +70,14 @@ module Rfm # } # # This code iterates through the rows of the _Orders_ portal. - # + # + # As a convenience, you can call a specific portal as a method on your record, if the table occurrence name does + # not have any characters that are prohibited in ruby method names, just as you can call a field with a method: + # + # myRecord.orders.each {|portal_row| + # puts portal_row["Order Number"] + # } + # # =Field Types and Ruby Types # # RFM automatically converts data from FileMaker into a Ruby object with the most reasonable type possible. The @@ -99,55 +107,50 @@ module Rfm # changes so you can tell if the Record object you're looking at is up-to-date as compared to another # copy of the same record class Record < Rfm::CaseInsensitiveHash - + + attr_accessor :layout #, :resultset attr_reader :record_id, :mod_id, :portals + def_delegators :layout, :db, :database, :server, :field_meta, :portal_meta, :field_names, :portal_names - def initialize(record, result, field_meta, layout, portal=nil) - @record_id = record['record-id'] - @mod_id = record['mod-id'] - @mods = {} - @layout = layout - @portals ||= Rfm::CaseInsensitiveHash.new - - relatedsets = !portal && result.instance_variable_get(:@include_portals) ? record.xpath('relatedset') : [] - - record.xpath('field').each do |field| - field_name = field['name'] - field_name.gsub!(Regexp.new(portal + '::'), '') if portal - datum = [] - - field.xpath('data').each do |x| - datum.push(field_meta[field_name].coerce(x.inner_text, result)) - end - - if datum.length == 1 - self[field_name] = datum[0] - elsif datum.length == 0 - self[field_name] = nil - else - self[field_name] = datum - end - end - - unless relatedsets.empty? - relatedsets.each do |relatedset| - tablename, records = relatedset['table'], [] - - relatedset.xpath('record').each do |record| - records << self.class.new(record, result, result.portal_meta[tablename], layout, tablename) - end - - @portals[tablename] = records - end + # This is called during the parsing process, but only to allow creation of the correct type of model instance. + # This is also called by the end-user when constructing a new empty record, but it is called from the model subclass. + def self.new(*args) # resultset + record = case + # Get model from layout, then allocate record. + # This should only use model class if the class already exists, + # since we don't want to create classes that aren't defined by the user - they won't be persistant. + when args[0].is_a?(Resultset) && args[0].layout && args[0].layout.model + args[0].layout.modelize.allocate + # Allocate instance of Rfm::Record. + else + self.allocate end - - @loaded = true + + record.send(:initialize, *args) + record + # rescue + # puts "Record.new bombed and is defaulting to super.new. Error: #{$!}" + # super end - def self.build_records(records, result, field_meta, layout, portal=nil) - records.each do |record| - result << self.new(record, result, field_meta, layout, portal) + def initialize(*args) # resultset, attributes + @mods ||= {} + @portals ||= Rfm::CaseInsensitiveHash.new + options = args.rfm_extract_options! + if args[0].is_a?(Resultset) + @layout = args[0].layout + elsif self.is_a?(Base) + @layout = self.class.layout + @layout.field_keys.each do |field| + self[field] = nil + end + self.update_attributes(options) unless options == {} + self.merge!(@mods) unless @mods == {} + @loaded = true end + _attach_as_instance_variables(args[1]) if args[1].is_a? Hash + #@loaded = true + self end # Saves local changes to the Record object back to Filemaker. For example: @@ -164,7 +167,8 @@ def self.build_records(records, result, field_meta, layout, portal=nil) # to optimize on your end. Just save, and if you've changed the record it will be saved. If not, no # server hit is incurred. def save - self.merge(@layout.edit(self.record_id, @mods)[0]) if @mods.size > 0 + # self.merge!(layout.edit(self.record_id, @mods)[0]) if @mods.size > 0 + self.replace_with_fresh_data(layout.edit(self.record_id, @mods)[0]) if @mods.size > 0 @mods.clear end @@ -172,10 +176,11 @@ def save # modified after the record was fetched but before it was saved. In other words, prevents you from # accidentally overwriting changes someone else made to the record. def save_if_not_modified - self.merge(@layout.edit(@record_id, @mods, {:modification_id => @mod_id})[0]) if @mods.size > 0 + # self.merge!(layout.edit(@record_id, @mods, {:modification_id => @mod_id})[0]) if @mods.size > 0 + self.replace_with_fresh_data(layout.edit(@record_id, @mods, {:modification_id => @mod_id})[0]) if @mods.size > 0 @mods.clear end - + # Gets the value of a field from the record. For example: # # first = myRecord["First Name"] @@ -189,31 +194,72 @@ def save_if_not_modified # # When you do, the change is noted, but *the data is not updated in FileMaker*. You must call # Record::save or Record::save_if_not_modified to actually save the data. - def []=(pname, value) - return super unless @loaded # keeps us from getting mods during initialization - name = pname - if self[name] != nil - @mods[name] = val + def [](key) + # Added by wbr, 2013-03-31 + return super unless @loaded + return fetch(key.to_s.downcase) + rescue IndexError + raise Rfm::ParameterError, "#{key} does not exists as a field in the current Filemaker layout." unless key.to_s == '' #unless (!layout or self.key?(key_string)) + end + + def respond_to?(symbol, include_private = false) + return true if self.include?(symbol.to_s) + super + end + + def []=(key, val) + key_string = key.to_s.downcase + key_string_base = key_string.split('.')[0] + return super unless @loaded # is this needed? yes, for loading fresh records. + unless self.key?(key_string) || (layout.field_keys.include?(key_string_base) rescue nil) + raise Rfm::ParameterError, "You attempted to modify a field (#{key_string}) that does not exist in the current Filemaker layout." + end + # @mods[key_string] = val + # TODO: This needs cleaning up. + # TODO: can we get field_type from record instead? + @mods[key_string] = if [Date, Time, DateTime].member?(val.class) + field_type = layout.field_meta[key_string.to_sym].result + case field_type + when 'time' + val.strftime(layout.time_format) + when 'date' + val.strftime(layout.date_format) + when 'timestamp' + val.strftime(layout.timestamp_format) + else + val + end else - raise Rfm::Error::ParameterError.new("You attempted to modify a field called '#{name}' on the Rfm::Record object, but that field does not exist.") + val end + super(key, val) + end + + def field_names + layout.field_names end - - def method_missing (symbol, *attrs) - # check for simple getter - return self[symbol.to_s] if self.include?(symbol.to_s) - - # check for setter - symbol_name = symbol.to_s - if symbol_name[-1..-1] == '=' && self.has_key?(symbol_name[0..-2]) - return @mods[symbol_name[0..-2]] = attrs[0] + + def replace_with_fresh_data(record) + self.replace record + [:@mod_id, :@record_id, :@portals, :@mods].each do |var| + self.instance_variable_set var, record.instance_variable_get(var) || {} end - super + self end - - def respond_to?(symbol, include_private = false) - return true if self[symbol.to_s] != nil + + + private + + def method_missing (symbol, *attrs, &block) + method = symbol.to_s + return self[method] if self.key?(method) + return @portals[method] if @portals and @portals.key?(method) + + if method =~ /(=)$/ + return self[$`] = attrs.first if self.key?($`) + end super end - end -end \ No newline at end of file + + end # Record +end # Rfm diff --git a/lib/rfm/resultset.rb b/lib/rfm/resultset.rb index 549e2130..f12dcf86 100644 --- a/lib/rfm/resultset.rb +++ b/lib/rfm/resultset.rb @@ -5,10 +5,9 @@ # Author:: Geoff Coffey (mailto:gwcoffey@gmail.com) # Copyright:: Copyright (c) 2007 Six Fried Rice, LLC and Mufaddal Khumri # License:: See MIT-LICENSE for details -require 'nokogiri' + require 'bigdecimal' require 'rfm/record' -require 'rfm/metadata/field' module Rfm @@ -37,12 +36,23 @@ module Rfm # it provides metadata about the portals in the ResultSet and the Fields on those portals class Resultset < Array - - attr_reader :layout - attr_reader :field_meta, :portal_meta - attr_reader :date_format, :time_format, :timestamp_format - attr_reader :total_count, :foundset_count - + include Config + + attr_reader :layout, :meta, :calling_object + # attr_reader :layout, :database, :server, :calling_object, :doc + # attr_reader :field_meta, :portal_meta, :include_portals, :datasource + # attr_reader :date_format, :time_format, :timestamp_format + # attr_reader :total_count, :foundset_count, :table + #def_delegators :layout, :db, :database + # alias_method :db, :database + def_delegators :meta, :field_meta, :portal_meta, :date_format, :time_format, :timestamp_format, :total_count, :foundset_count, :fetch_size, :table, :error, :field_names, :field_keys, :portal_names + + class << self + def load_data(data, object=self.new) + Rfm::SaxParser.parse(data, :fmresultset, object) + end + end + # Initializes a new ResultSet object. You will probably never do this your self (instead, use the Layout # object to get various ResultSet obejects). # @@ -64,73 +74,69 @@ class Resultset < Array # * *portals* is a hash (with table occurrence names for keys and Field objects for values). If your # layout contains portals, you can find out what fields they contain here. Again, if it's the data you're # after, you want to look at the Record object. - - def initialize(server, xml_response, layout, portals=nil) - @layout = layout - @field_meta ||= Rfm::CaseInsensitiveHash.new - @portal_meta ||= Rfm::CaseInsensitiveHash.new - @include_portals = portals - - doc = Nokogiri.XML(remove_namespace(xml_response)) - - error = doc.xpath('/fmresultset/error').attribute('code').value.to_i - check_for_errors(error, server.state[:raise_on_401]) - - datasource = doc.xpath('/fmresultset/datasource') - meta = doc.xpath('/fmresultset/metadata') - resultset = doc.xpath('/fmresultset/resultset') - - @date_format = convert_date_time_format(datasource.attribute('date-format').value) - @time_format = convert_date_time_format(datasource.attribute('time-format').value) - @timestamp_format = convert_date_time_format(datasource.attribute('timestamp-format').value) - - @foundset_count = resultset.attribute('count').value.to_i - @total_count = datasource.attribute('total-count').value.to_i - - parse_fields(meta) - parse_portals(meta) if @include_portals - - Rfm::Record.build_records(resultset.xpath('record'), self, @field_meta, @layout) - + def initialize(*args) # parent, layout + config(*args) + self.meta + end # initialize + + def config(*args) + super do |params| + (@layout = params[:objects][0]) if params && + params[:objects] && + params[:objects][0] && + params[:objects][0].is_a?(Rfm::Layout) + end + end + + # This method was added for situations where a layout was not provided at resultset instantiation, + # such as when loading a resultset from an xml file. + def layout + @layout ||= (Layout.new(meta.layout, self) if meta.respond_to? :layout) + end + + def database + layout.database + end + + alias_method :db, :database + + def server + database.server end - + + def meta + # Access the meta inst var here. + @meta ||= Metadata::ResultsetMeta.new + end + + # Deprecated on 7/29/2014. Stop using. + def handle_new_record(attributes) + r = Rfm::Record.new(self, attributes, {}) + self << r + r + end + + def end_datasource_element_callback(cursor) + %w(date_format time_format timestamp_format).each{|f| convert_date_time_format(send(f))} + @meta.attach_layout_object_from_cursor(cursor) + end + private - def remove_namespace(xml) - xml.gsub('xmlns="http://www.filemaker.com/xml/fmresultset" version="1.0"', '') - end - - def check_for_errors(code, raise_401) - raise Rfm::Error.getError(code) if code != 0 && (code != 401 || raise_401) - end - - def parse_fields(meta) - meta.xpath('field-definition').each do |field| - @field_meta[field['name']] = Rfm::Metadata::Field.new(field) - end - end - def parse_portals(meta) - meta.xpath('relatedset-definition').each do |relatedset| - table, fields = relatedset.attribute('table').value, {} + def check_for_errors(error_code=@meta['error'].to_i, raise_401=state[:raise_401]) + #puts ["\nRESULTSET#check_for_errors", "meta[:error] #{@meta[:error]}", "error_code: #{error_code}", "raise_401: #{raise_401}"] + raise Rfm::Error.getError(error_code) if error_code != 0 && (error_code != 401 || raise_401) + end - relatedset.xpath('field-definition').each do |field| - name = field.attribute('name').value.gsub(Regexp.new(table + '::'), '') - fields[name] = Rfm::Metadata::Field.new(field) - end + def convert_date_time_format(fm_format) + fm_format.gsub!('MM', '%m') + fm_format.gsub!('dd', '%d') + fm_format.gsub!('yyyy', '%Y') + fm_format.gsub!('HH', '%H') + fm_format.gsub!('mm', '%M') + fm_format.gsub!('ss', '%S') + fm_format + end - @portal_meta[table] = fields - end - end - - def convert_date_time_format(fm_format) - fm_format.gsub!('MM', '%m') - fm_format.gsub!('dd', '%d') - fm_format.gsub!('yyyy', '%Y') - fm_format.gsub!('HH', '%H') - fm_format.gsub!('mm', '%M') - fm_format.gsub!('ss', '%S') - fm_format - end - end -end \ No newline at end of file +end diff --git a/lib/rfm/server.rb b/lib/rfm/server.rb index 367b0d61..ea5561df 100644 --- a/lib/rfm/server.rb +++ b/lib/rfm/server.rb @@ -1,5 +1,6 @@ require 'net/https' require 'cgi' + module Rfm # This class represents a single FileMaker server. It is initialized with basic # connection information, including the hostname, port number, and default database @@ -45,9 +46,9 @@ module Rfm # * *host_name* is the host name this server points to # * *port* is the port number this server communicates on # * *state* is a hash of all server options used to initialize this server - - - + + + # The Database object represents a single FileMaker Pro database. When you retrieve a Database # object from a server, its account name and password are set to the account name and password you # used when initializing the Server object. You can override this of course: @@ -107,13 +108,15 @@ module Rfm # * *name* is the name of this database # * *state* is a hash of all server options used to initialize this server class Server - # + include Config + + # To create a Server object, you typically need at least a host name: # # myServer = Rfm::Server.new({:host => 'my.host.com'}) # - # Several other options are supported: - # + # ===Several other options are supported + # # * *host* the hostname of the Web Publishing Engine (WPE) server (defaults to 'localhost') # # * *port* the port number the WPE is listening no (defaults to 80 unless *ssl* +true+ which sets it to 443) @@ -138,13 +141,13 @@ class Server # ignores FileMaker's 401 error (no records found) and returns an empty record set instead; if you # prefer a raised error when a find produces no errors, set this option to +true+ # - #SSL Options (SSL AND CERTIFICATE VERIFICATION ARE ON BY DEFAULT): - # + # ===SSL Options (SSL AND CERTIFICATE VERIFICATION ARE ON BY DEFAULT) + # # * *ssl* +false+ if you want to turn SSL (HTTPS) off when connecting to connect to FileMaker (default is +true+) # - # If you are using SSL and want to verify the certificate use the following options: + # If you are using SSL and want to verify the certificate, use the following options: # - # * *root_cert* +false+ if you do not want to verify your SSL session (default is +true+). + # * *root_cert* +true+ is the default. If you do not want to verify your SSL session, set this to +false+. # You will want to turn this off if you are using a self signed certificate and do not have a certificate authority cert file. # If you choose this option you will need to provide a cert *root_cert_name* and *root_cert_path* (if not in root directory). # @@ -154,8 +157,8 @@ class Server # # * *root_cert_path* path to cert file. (defaults to '/' if no path given) # - #Configuration Examples: - # + # ===Configuration Examples + # # Example to turn off SSL: # # myServer = Rfm::Server.new({ @@ -192,32 +195,12 @@ class Server # :root_cert_name => 'example.pem' # :root_cert_path => '/usr/cert_file/' # }) - - def initialize(options) - @state = { - :host => 'localhost', - :port => 80, - :ssl => true, - :root_cert => true, - :root_cert_name => '', - :root_cert_path => '/', - :account_name => '', - :password => '', - :log_actions => false, - :log_responses => false, - :warn_on_redirect => true, - :raise_on_401 => false - }.merge(options) - - @state.freeze - - @host_name = @state[:host] - @scheme = @state[:ssl] ? "https" : "http" - @port = @state[:ssl] && options[:port].nil? ? 443 : @state[:port] - - @db = Rfm::Factory::DbFactory.new(self) + def initialize(*args) + config(*args) + raise Rfm::Error::RfmError.new(0, "New instance of Rfm::Server has no host name. Attempted name '#{state[:host]}'.") if state[:host].to_s == '' + @databases = Rfm::Factory::DbFactory.new(self) end - + # Access the database object representing a database on the server. For example: # # myServer['Customers'] @@ -230,12 +213,32 @@ def initialize(options) # get no error at this point if the database you access doesn't exist. Instead, you'll # receive an error when you actually try to perform some action on a layout from this # database. - def [](dbname) - self.db[dbname] + # def [](dbname, acnt=nil, pass=nil) + # self.db[dbname, acnt, pass] + # end + def_delegator :databases, :[] + + attr_reader :databases #, :host_name, :port, :scheme, :state + # Legacy Rfm method to get/create databases from server object + alias_method :db, :databases + + def config(*args) + super(:capture_strings_with=>[:host, :account_name, :password]) + super(*args) + end + + def host_name + state[:host] + end + + def scheme + state[:ssl] ? "https" : "http" end - - attr_reader :db, :host_name, :port, :scheme, :state - + + def port + state[:ssl] && state[:port].nil? ? 443 : state[:port] + end + # Performs a raw FileMaker action. You will generally not call this method directly, but it # is exposed in case you need to do something "under the hood." # @@ -263,125 +266,6 @@ def [](dbname) # }, # { :max_records => 20 } # ) - def connect(account_name, password, action, args, options = {}) - post = args.merge(expand_options(options)).merge({action => ''}) - http_fetch(@host_name, @port, "/fmi/xml/fmresultset.xml", account_name, password, post) - end - - def load_layout(layout) - post = {'-db' => layout.db.name, '-lay' => layout.name, '-view' => ''} - http_fetch(@host_name, @port, "/fmi/xml/FMPXMLLAYOUT.xml", layout.db.account_name, layout.db.password, post) - end - - private - - def http_fetch(host_name, port, path, account_name, password, post_data, limit=10) - raise Rfm::CommunicationError.new("While trying to reach the Web Publishing Engine, RFM was redirected too many times.") if limit == 0 - - if @state[:log_actions] == true - qs = post_data.collect{|key,val| "#{CGI::escape(key.to_s)}=#{CGI::escape(val.to_s)}"}.join("&") - warn "#{@scheme}://#{@host_name}:#{@port}#{path}?#{qs}" - end - - request = Net::HTTP::Post.new(path) - request.basic_auth(account_name, password) - request.set_form_data(post_data) - - response = Net::HTTP.new(host_name, port) - - if @state[:ssl] - response.use_ssl = true - if @state[:root_cert] - response.verify_mode = OpenSSL::SSL::VERIFY_PEER - response.ca_file = File.join(@state[:root_cert_path], @state[:root_cert_name]) - else - response.verify_mode = OpenSSL::SSL::VERIFY_NONE - end - end - - response = response.start { |http| http.request(request) } - if @state[:log_responses] == true - response.to_hash.each { |key, value| warn "#{key}: #{value}" } - warn response.body - end - - case response - when Net::HTTPSuccess - response - when Net::HTTPRedirection - if @state[:warn_on_redirect] - warn "The web server redirected to " + response['location'] + - ". You should revise your connection hostname or fix your server configuration if possible to improve performance." - end - newloc = URI.parse(response['location']) - http_fetch(newloc.host, newloc.port, newloc.request_uri, account_name, password, post_data, limit - 1) - when Net::HTTPUnauthorized - msg = "The account name (#{account_name}) or password provided is not correct (or the account doesn't have the fmxml extended privilege)." - raise Rfm::AuthenticationError.new(msg) - when Net::HTTPNotFound - msg = "Could not talk to FileMaker because the Web Publishing Engine is not responding (server returned 404)." - raise Rfm::CommunicationError.new(msg) - else - msg = "Unexpected response from server: #{response.code} (#{response.class.to_s}). Unable to communicate with the Web Publishing Engine." - raise Rfm::CommunicationError.new(msg) - end - end - - def expand_options(options) - result = {} - options.each do |key,value| - case key - when :max_records - result['-max'] = value - when :skip_records - result['-skip'] = value - when :sort_field - if value.kind_of? Array - raise Rfm::ParameterError.new(":sort_field can have at most 9 fields, but you passed an array with #{value.size} elements.") if value.size > 9 - value.each_index { |i| result["-sortfield.#{i+1}"] = value[i] } - else - result["-sortfield.1"] = value - end - when :sort_order - if value.kind_of? Array - raise Rfm::ParameterError.new(":sort_order can have at most 9 fields, but you passed an array with #{value.size} elements.") if value.size > 9 - value.each_index { |i| result["-sortorder.#{i+1}"] = value[i] } - else - result["-sortorder.1"] = value - end - when :post_script - if value.class == Array - result['-script'] = value[0] - result['-script.param'] = value[1] - else - result['-script'] = value - end - when :pre_find_script - if value.class == Array - result['-script.prefind'] = value[0] - result['-script.prefind.param'] = value[1] - else - result['-script.presort'] = value - end - when :pre_sort_script - if value.class == Array - result['-script.presort'] = value[0] - result['-script.presort.param'] = value[1] - else - result['-script.presort'] = value - end - when :response_layout - result['-lay.response'] = value - when :logical_operator - result['-lop'] = value - when :modification_id - result['-modid'] = value - else - raise Rfm::ParameterError.new("Invalid option: #{key} (are you using a string instead of a symbol?)") - end - end - return result - end - + end -end \ No newline at end of file +end diff --git a/lib/rfm/utilities/case_insensitive_hash.rb b/lib/rfm/utilities/case_insensitive_hash.rb index 73266e5d..14902f7f 100644 --- a/lib/rfm/utilities/case_insensitive_hash.rb +++ b/lib/rfm/utilities/case_insensitive_hash.rb @@ -1,10 +1,13 @@ module Rfm class CaseInsensitiveHash < Hash + def []=(key, value) - super(key.downcase, value) + super(key.to_s.downcase, value) end + def [](key) - super(key.downcase) + super(key.to_s.downcase) end + end -end \ No newline at end of file +end diff --git a/lib/rfm/utilities/compound_query.rb b/lib/rfm/utilities/compound_query.rb new file mode 100644 index 00000000..57f6a1dd --- /dev/null +++ b/lib/rfm/utilities/compound_query.rb @@ -0,0 +1,115 @@ +# The classes in this module are used internally by RFM and are not intended for outside use. +module Rfm + + # Class to build complex FMP queries. + # Perform Filemaker find using complex boolean logic (multiple value options for a single field), + # or create multiple find requests. + # Also allow find requests to be :omit. + class CompoundQuery < Array + + attr_accessor :original_input, :query_type, :key_values, :key_arrays, :key_map, :key_map_string, :key_counter, :field_mapping + + def self.build_test + new([{:field1=>['val1a','val1b','val1c'], :field2=>'val2'},{:omit=>true, :field3=>'val3', :field4=>'val4'}, {:omit=>true, :field5=>['val5a','val5b'], :field6=>['val6a','val6b']}], {}) + end + + # New CompoundQuery objects expect one of 3 data types: + # * string/integer representing FMP internal record id + # * hash of find-criteria + # * array of find-criteria hashes + # + # Returns self as ['-fmpaction', {:hash=>'of', :key=>'values'}, {:options=>'hash'}] + def initialize(query, options={}) + @options = options + @field_mapping = options.delete(:field_mapping) || {} + @original_input = query + @key_values = {} + @key_arrays = [] + @key_map = [] + @key_map_string = '' + @key_counter = 0 + + case query + when Hash + if query.detect{|k,v| v.kind_of? Array or k == :omit} + @query_type = 'mixed' + else + @query_type = 'standard' + end + when Array + @query_type = 'compound' + else + @query_type = 'recid' + end + build_query + end + + # Master control method to build output + def build_query(input=original_input) + case @query_type + when 'mixed', 'compound' + input.rfm_force_array.each do |hash| + build_key_map(build_key_values(hash)) + end + translate_key_map + self.push '-findquery' + self.push @key_values.merge('-query'=>@key_map_string) + when 'standard' + self.push '-find' + self.push @original_input + when 'recid' + self.push '-find' + self.push '-recid' => @original_input.to_s + end + self.push @options + self + end + + + # Build key-value definitions and query map '-q1...'. + # Converts query_hash to fmresultset uri format for -findquery query type. + def build_key_values(input_hash) + input_hash = input_hash.clone + keyarray = [] + omit = input_hash.delete(:omit) + input_hash.each do |key,val| + query_tag = [] + val = val.rfm_force_array + val << nil if val.empty? + val.each do |v| + @key_values["-q#{key_counter}"] = field_mapping[key] || key + @key_values["-q#{key_counter}.value"] = v + query_tag << "q#{key_counter}" + @key_counter += 1 + end + keyarray << query_tag + end + (keyarray << :omit) if omit + @key_arrays << keyarray + keyarray + end + + + # Input array of arrays. + # Transform single key_array into key_map (array of requests). + # Creates all combinations of sub-arrays where each combination contains one element of each subarray. + def build_key_map(key_array) + key_array = key_array.clone + omit = key_array.delete(:omit) + len = key_array.length + flat = key_array.flatten + rslt = flat.combination(len).select{|c| key_array.all?{|a| (a & c).size > 0}}.each{|c| c.unshift(:omit) if omit} + @key_map.concat rslt + end + + # Translate @key_map to FMP -query string + def translate_key_map(keymap=key_map) + keymap = keymap.clone + inner = keymap.collect {|a| "#{'!' if a.delete(:omit)}(#{a.join(',')})"} + outter = inner.join(';') + @key_map_string << outter + end + + end # CompoundQuery + +end # Rfm diff --git a/lib/rfm/utilities/config.rb b/lib/rfm/utilities/config.rb new file mode 100644 index 00000000..08e2b17f --- /dev/null +++ b/lib/rfm/utilities/config.rb @@ -0,0 +1,243 @@ +module Rfm + # + # Top level config hash accepts any defined config parameters, + # or group-name keys pointing to config subsets. + # The subsets can be any grouping of defined config parameters, as a hash. + # See CONFIG_KEYS for defined config parameters. + # + module Config + require 'yaml' + require 'erb' + + CONFIG_KEYS = %w( + file_name + file_path + parser + host + port + proxy + account_name + password + database + layout + ignore_bad_data + ssl + root_cert + root_cert_name + root_cert_path + warn_on_redirect + raise_on_401 + timeout + log_actions + log_responses + log_parser + use + parent + template + grammar + field_mapping + capture_strings_with + logger + decimal_separator + ) + + CONFIG_DONT_STORE = %w(strings using parents symbols objects) #capture_strings_with) + + extend self + @config = {} + + # Set @config with args & options hash. + # Args should be symbols representing configuration groups, + # with optional config hash as last arg, to be merged on top. + # Returns @config. + # + # == Sets @config with :use => :group1, :layout => 'my_layout' + # config :group1, :layout => 'my_layout + # + # Factory.server, Factory.database, Factory.layout, and Base.config can take + # a string as the first argument, refering to the relevent server/database/layout name. + # + # == Pass a string as the first argument, to be used in the immediate context + # config 'my_layout' # in the model, to set model configuration + # Factory.layout 'my_layout', :my_group # to get a layout from settings in :my_group + # + def config(*args, &block) + @config ||= {} + return @config if args.empty? + config_write(*args, &block) + @config + end + + # Sets @config just as above config method, but clears @config first. + def config_clear(*args) + @config = {} + return @config if args.empty? + config_write(*args) + @config + end + + # Reads compiled config, including filters and ad-hoc configuration params passed in. + # If first n parameters are strings, they will be appended to config[:strings]. + # If next n parameters are symbols, they will be used to filter the result. These + # filters will override all stored config[:use] settings. + # The final optional hash should be ad-hoc config settings. + # + # == Gets top level settings, merged with group settings, merged with local and ad-hoc settings. + # get_config :my_server_group, :layout => 'my_layout' # This gets top level settings, + # + # == Gets top level settings, merged with local and ad-hoc settings. + # get_config :layout => 'my_layout + # + def get_config(*arguments) + #puts caller_locations(1,1)[0] + args = arguments.clone + @config ||= {} + options = config_extract_options!(*args) + strings = options[:strings].rfm_force_array || [] + symbols = options[:symbols].rfm_force_array.concat(options[:hash][:use].rfm_force_array) || [] + objects = options[:objects].rfm_force_array || [] + + rslt = config_merge_with_parent(symbols).merge(options[:hash]) + #using = rslt[:using].rfm_force_array + sanitize_config(rslt, CONFIG_DONT_STORE, false) + rslt[:using].delete "" + rslt[:parents].delete "" + rslt.merge(:strings=>strings, :objects=>objects) + end + + def state(*args) + return @_state if args.empty? && !@state.nil? && (RUBY_VERSION[0,1].to_i > 1 ? (caller_locations(1,1) == @_last_state_caller) : false) + @_state = get_config(*args) + (@_last_state_caller = caller_locations(1,1)) if RUBY_VERSION[0,1].to_i > 1 + @_state + end + + def log + Rfm.log + end + + protected + + # Get or load a config file as the top-level config (above RFM_CONFIG constant). + # Default file name is rfm.yml. + # Default paths are '' and 'config/'. + # File name & paths can be set in RFM_CONFIG and Rfm.config. + # Change file name with :file_name => 'something.else' + # Change file paths with :file_path => ['array/of/', 'file/paths/'] + def get_config_file + @@config_file_data ||= ( + config_file_name = @config[:file_name] || (RFM_CONFIG[:file_name] rescue nil) || 'rfm.yml' + config_file_paths = [''] | [(@config[:file_path] || (RFM_CONFIG[:file_path] rescue nil) || %w( config/ ./ ))].flatten + config_file_paths.collect do |path| + (YAML.load(ERB.new(File.read(File.join(path, config_file_name))).result) rescue {}) + end.inject({}){|h,a| h.merge(a)} + ) || {} + end + + # Get the top-level configuration from yml file and RFM_CONFIG + def get_config_base + get_config_file.merge((defined?(RFM_CONFIG) and RFM_CONFIG.is_a?(Hash)) ? RFM_CONFIG : {}) + end + + # Get the parent configuration object according to @config[:parent] + def get_config_parent + @config ||= {} + case + when @config[:parent].is_a?(String) + eval(@config[:parent]) + when !@config[:parent].nil? && @config[:parent].is_a?(Rfm::Config) + @config[:parent] + else + eval('Rfm::Config') + end + end + + # Merge args into @config, as :use=>[arg1, arg2, ...] + # Then merge optional config hash into @config. + # Pass in a block to use with parsed config in args. + def config_write(*args) #(opt, args) + options = config_extract_options!(*args) + options[:symbols].each{|a| @config.merge!(:use=>a.to_sym){|h,v1,v2| [v1].flatten << v2 }} + @config.merge!(options[:hash]).reject! {|k,v| CONFIG_DONT_STORE.include? k.to_s} + #options[:hash][:capture_strings_with].rfm_force_array.each do |label| + @config[:capture_strings_with].rfm_force_array.each do |label| + string = options[:strings].delete_at(0) + (@config[label] = string) if string && !string.empty? + end + parent = (options[:objects].delete_at(0) || options[:hash][:parent]) + (@config[:parent] = parent) if parent + yield(options) if block_given? + end + + # Get composite config from all levels, processing :use parameters at each level + def config_merge_with_parent(filters=nil) + @config ||= {} + + # Get upstream compilation + upstream = if (self != Rfm::Config) + #puts [self, @config[:parent], get_config_parent].join(', ') + get_config_parent.config_merge_with_parent + else + get_config_base + end.clone + + upstream[:using] ||= [] + upstream[:parents] ||= ['file', 'RFM_CONFIG'] + + filters = (@config[:use].rfm_force_array | filters.rfm_force_array).compact + + rslt = config_filter(upstream, filters).merge(config_filter(@config, filters)) + + rslt[:using].concat((@config[:use].rfm_force_array | filters).compact.flatten) #.join + rslt[:parents] << @config[:parent].to_s + + rslt.delete :parent + + rslt || {} + # rescue + # puts "Config#config_merge_with_parent for '#{self.class}' falied with #{$1}" + end + + # Returns a configuration hash overwritten by :use filters in the hash + # that match passed-in filter names or any filter names contained within the hash's :use key. + def config_filter(conf, filters=nil) + conf = conf.clone + filters = (conf[:use].rfm_force_array | filters.rfm_force_array).compact + if (!filters.nil? && !filters.empty?) + filters.each do |f| + next unless conf[f] + conf.merge!(conf[f] || {}) + end + end + conf.delete(:use) + conf + end + + # Extract arguments into strings, symbols, objects, hash. + def config_extract_options!(*args) + strings, symbols, objects = [], [], [] + options = args.last.is_a?(Hash) ? args.pop : {} + args.each do |a| + case + when a.is_a?(String) + strings << a + when a.is_a?(Symbol) + symbols << a + else + objects << a + end + end + {:strings=>strings, :symbols=>symbols, :objects=>objects, :hash=>options} + end + + # Remove un-registered keys from a configuration hash. + # Keep should be a list of strings representing keys to keep. + def sanitize_config(conf={}, keep=[], dupe=false) + (conf = conf.clone) if dupe + conf.reject!{|k,v| (!CONFIG_KEYS.include?(k.to_s) or [{},[],''].include?(v)) and !keep.include? k.to_s } + conf + end + + end # module Config + +end # module Rfm diff --git a/lib/rfm/utilities/connection.rb b/lib/rfm/utilities/connection.rb new file mode 100644 index 00000000..0ba83859 --- /dev/null +++ b/lib/rfm/utilities/connection.rb @@ -0,0 +1,230 @@ +# Connection object takes over the communication functionality that was previously in Rfm::Server. +# TODO: Clean up the way :grammar is sent in to the initializing method. +# Currently, the actual connection instance config doesn't get set with the correct grammar, +# even if the http_fetch is using the correct grammar. + +require 'net/https' +require 'cgi' + +module Rfm + # These have been moved to rfm.rb. + # SaxParser.default_class = CaseInsensitiveHash + # SaxParser.template_prefix = File.join(File.dirname(__FILE__), './sax/') + # SaxParser.templates.merge!({ + # :fmpxmllayout => 'fmpxmllayout.yml', + # :fmresultset => 'fmresultset.yml', + # :fmpxmlresult => 'fmpxmlresult.yml', + # :none => nil + # }) + + class Connection + include Config + + def initialize(action, params, request_options={}, *args) + config(*args) + + # Action sent to FMS + @action = action + # Query params sent to FMS + @params = params + # Additional options sent to FMS + @request_options = request_options + + @defaults = { + :host => 'localhost', + #:port => 80, + :proxy=>false, # array of (p_addr, p_port = nil, p_user = nil, p_pass = nil) + :ssl => true, + :root_cert => true, + :root_cert_name => '', + :root_cert_path => '/', + :account_name => '', + :password => '', + :log_actions => false, + :log_responses => false, + :log_parser => false, + :warn_on_redirect => true, + :raise_on_401 => false, + :timeout => 60, + :ignore_bad_data => false, + :template => :fmresultset, + :grammar => 'fmresultset' + } #.merge(options) + end + + def state(*args) + @defaults.merge(super(*args)) + end + + def host_name + state[:host] + end + + def scheme + state[:ssl] ? "https" : "http" + end + + def port + state[:ssl] && state[:port].nil? ? 443 : state[:port] + end + + def connect(action=@action, params=@params, request_options = @request_options, account_name=state[:account_name], password=state[:password]) + grammar_option = request_options.delete(:grammar) + post = params.merge(expand_options(request_options)).merge({action => ''}) + grammar = select_grammar(post, :grammar=>grammar_option) + http_fetch(host_name, port, "/fmi/xml/#{grammar}.xml", account_name, password, post) + end + + def select_grammar(post, options={}) + grammar = state(options)[:grammar] || 'fmresultset' + if grammar.to_s.downcase == 'auto' + # TODO: Build grammar parser in new sax engine templates to handle FMPXMLRESULT. + return "fmresultset" + # post.keys.find(){|k| %w(-find -findall -dbnames -layoutnames -scriptnames).include? k.to_s} ? "FMPXMLRESULT" : "fmresultset" + else + grammar + end + end + + def parse(template=nil, initial_object=nil, parser=nil, options={}) + template ||= state[:template] + #(template = 'fmresultset.yml') unless template + #(template = File.join(File.dirname(__FILE__), '../sax/', template)) if template.is_a? String + Rfm::SaxParser.parse(connect.body, template, initial_object, parser, state(*options)).result + end + + + + + private + + def http_fetch(host_name, port, path, account_name, password, post_data, limit=10) + raise Rfm::CommunicationError.new("While trying to reach the Web Publishing Engine, RFM was redirected too many times.") if limit == 0 + + if state[:log_actions] == true + #qs = post_data.collect{|key,val| "#{CGI::escape(key.to_s)}=#{CGI::escape(val.to_s)}"}.join("&") + qs_unescaped = post_data.collect{|key,val| "#{key.to_s}=#{val.to_s}"}.join("&") + #warn "#{@scheme}://#{@host_name}:#{@port}#{path}?#{qs}" + log.info "#{scheme}://#{host_name}:#{port}#{path}?#{qs_unescaped}" + end + + request = Net::HTTP::Post.new(path) + request.basic_auth(account_name, password) + request.set_form_data(post_data) + + if state[:proxy] + connection = Net::HTTP::Proxy(*state[:proxy]).new(host_name, port) + else + connection = Net::HTTP.new(host_name, port) + end + #ADDED LONG TIMEOUT TIMOTHY TING 05/12/2011 + connection.open_timeout = connection.read_timeout = state[:timeout] + if state[:ssl] + connection.use_ssl = true + if state[:root_cert] + connection.verify_mode = OpenSSL::SSL::VERIFY_PEER + connection.ca_file = File.join(state[:root_cert_path], state[:root_cert_name]) + else + connection.verify_mode = OpenSSL::SSL::VERIFY_NONE + end + end + + response = connection.start { |http| http.request(request) } + if state[:log_responses] == true + response.to_hash.each { |key, value| log.info "#{key}: #{value}" } + log.info response.body + end + + case response + when Net::HTTPSuccess + response + when Net::HTTPRedirection + if state[:warn_on_redirect] + log.warn "The web server redirected to " + response['location'] + + ". You should revise your connection hostname or fix your server configuration if possible to improve performance." + end + newloc = URI.parse(response['location']) + http_fetch(newloc.host, newloc.port, newloc.request_uri, account_name, password, post_data, limit - 1) + when Net::HTTPUnauthorized + msg = "The account name (#{account_name}) or password provided is not correct (or the account doesn't have the fmxml extended privilege)." + raise Rfm::AuthenticationError.new(msg) + when Net::HTTPNotFound + msg = "Could not talk to FileMaker because the Web Publishing Engine is not responding (server returned 404)." + raise Rfm::CommunicationError.new(msg) + else + msg = "Unexpected response from server: #{response.code} (#{response.class.to_s}). Unable to communicate with the Web Publishing Engine." + raise Rfm::CommunicationError.new(msg) + end + end + + def expand_options(options) + result = {} + field_mapping = options.delete(:field_mapping) || {} + options.each do |key,value| + case key.to_sym + when :max_portal_rows + result['-relatedsets.max'] = value + result['-relatedsets.filter'] = 'layout' + when :ignore_portals + result['-relatedsets.max'] = 0 + result['-relatedsets.filter'] = 'layout' + when :max_records + result['-max'] = value + when :skip_records + result['-skip'] = value + when :sort_field + if value.kind_of? Array + raise Rfm::ParameterError.new(":sort_field can have at most 9 fields, but you passed an array with #{value.size} elements.") if value.size > 9 + value.each_index { |i| result["-sortfield.#{i+1}"] = field_mapping[value[i]] || value[i] } + else + result["-sortfield.1"] = field_mapping[value] || value + end + when :sort_order + if value.kind_of? Array + raise Rfm::ParameterError.new(":sort_order can have at most 9 fields, but you passed an array with #{value.size} elements.") if value.size > 9 + value.each_index { |i| result["-sortorder.#{i+1}"] = value[i] } + else + result["-sortorder.1"] = value + end + when :post_script + if value.class == Array + result['-script'] = value[0] + result['-script.param'] = value[1] + else + result['-script'] = value + end + when :pre_find_script + if value.class == Array + result['-script.prefind'] = value[0] + result['-script.prefind.param'] = value[1] + else + result['-script.presort'] = value + end + when :pre_sort_script + if value.class == Array + result['-script.presort'] = value[0] + result['-script.presort.param'] = value[1] + else + result['-script.presort'] = value + end + when :response_layout + result['-lay.response'] = value + when :logical_operator + result['-lop'] = value + when :modification_id + result['-modid'] = value + else + if state.keys.member?(key.to_sym) + state(key.to_sym=>value) + else + raise Rfm::ParameterError.new("Invalid option: #{key} (are you using a string instead of a symbol?)") + end + end + end + return result + end + + end # Connection + + +end # Rfm diff --git a/lib/rfm/utilities/core_ext.rb b/lib/rfm/utilities/core_ext.rb new file mode 100644 index 00000000..23a02de0 --- /dev/null +++ b/lib/rfm/utilities/core_ext.rb @@ -0,0 +1,171 @@ +require 'forwardable' + +Module.module_eval do + # Adds ability to forward methods to other objects using 'def_delegator' + include Forwardable +end + +class Object + + #extend Forwardable + + # Adds methods to put instance variables in rfm_metaclass, plus getter/setters + # This is useful to hide instance variables in objects that would otherwise show "too much" information. + def self.meta_attr_accessor(*names) + meta_attr_reader(*names) + meta_attr_writer(*names) + end + + def self.meta_attr_reader(*names) + names.each do |n| + define_method(n.to_s) {rfm_metaclass.instance_variable_get("@#{n}")} + end + end + + def self.meta_attr_writer(*names) + names.each do |n| + define_method(n.to_s + "=") {|val| rfm_metaclass.instance_variable_set("@#{n}", val)} + end + end + + # Wrap an object in Array, if not already an Array, + # since XmlMini doesn't know which will be returnd for any particular element. + # See Rfm Layout & Record where this is used. + def rfm_force_array + return [] if self.nil? + self.is_a?(Array) ? self : [self] + end + + # Just testing this functionality + def rfm_local_methods + self.methods - self.class.superclass.methods + end + + private + + # Like singleton_method or 'metaclass' from ActiveSupport. + def rfm_metaclass + class << self + self + end + end + + # Get the superclass object of self. + def rfm_super + SuperProxy.new(self) + end + +end # Object + + +class Array + # Taken from ActiveSupport extract_options!. + def rfm_extract_options! + last.is_a?(::Hash) ? pop : {} + end + + # These methods allow dynamic extension of array members with other modules. + # These methods also carry the @root object for reference, when you don't have the + # root object explicity referenced anywhere. + # + # These methods might slow down array traversal, as + # they add interpreted code to methods that were otherwise pure C. + def rfm_extend_members(klass, caller=nil) + @parent = caller + @root = caller.instance_variable_get(:@root) + @member_extension = klass + self.instance_eval do + class << self + attr_accessor :parent + + alias_method 'old_reader', '[]' + def [](*args) + member = old_reader(*args) + rfm_extend_member(member, @member_extension, args[0]) if args[0].is_a? Integer + member + end + + alias_method 'old_each', 'each' + def each + i = -1 + old_each do |member| + i = i + 1 + rfm_extend_member(member, @member_extension, i) + yield(member) + end + end + end + end unless defined? old_reader + self + end + + def rfm_extend_member(member, extension, i=nil) + if member and extension + unless member.instance_variable_get(:@root) + member.instance_variable_set(:@root, @root) + member.instance_variable_set(:@parent, self) + member.instance_variable_set(:@index, i) + member.instance_eval(){def root; @root; end} + member.instance_eval(){def parent; @parent; end} + member.instance_eval(){def get_index; @index; end} + end + member.extend(extension) + end + end + +end # Array + +class Hash + # TODO: Possibly deprecated, delete if not used. + def rfm_only(*keepers) + self.dup.each_key {|k| self.delete(k) if !keepers.include?(k)} + end + + def rfm_filter(*args) + options = args.rfm_extract_options! + delete = options[:delete] + self.dup.each_key do |k| + self.delete(k) if (delete ? args.include?(k) : !args.include?(k)) + end + end + + # Convert hash to Rfm::CaseInsensitiveHash + def to_cih + new = Rfm::CaseInsensitiveHash.new + self.each{|k,v| new[k] = v} + new + end +end # Hash + +# Allows access to superclass object +class SuperProxy + def initialize(obj) + @obj = obj + end + + def method_missing(meth, *args, &blk) + @obj.class.superclass.instance_method(meth).bind(@obj).call(*args, &blk) + end +end # SuperProxy + + +class Time + # Returns array of [date,time] in format suitable for FMP. + def to_fm_components(reset_time_if_before_today=false) + d = self.strftime('%m/%d/%Y') + t = if (Date.parse(self.to_s) < Date.today) and reset_time_if_before_today==true + "00:00:00" + else + self.strftime('%T') + end + [d,t] + end +end # Time + +class String + def title_case + self.gsub(/\w+/) do |word| + word.capitalize + end + end +end # String diff --git a/lib/rfm/utilities/factory.rb b/lib/rfm/utilities/factory.rb index 38fca4c9..88bfc113 100644 --- a/lib/rfm/utilities/factory.rb +++ b/lib/rfm/utilities/factory.rb @@ -5,80 +5,193 @@ # Copyright:: Copyright (c) 2007 Six Fried Rice, LLC and Mufaddal Khumri # License:: See MIT-LICENSE for details + module Rfm - module Factory # :nodoc: all - class DbFactory < Rfm::CaseInsensitiveHash - + + module Factory + # Acquired from Rfm::Base + @models ||= [] + + extend Config + config :parent=>'Rfm::Config' + + class ServerFactory < Rfm::CaseInsensitiveHash + def [](*args) + options = Factory.get_config(*args) + host = options[:strings].delete_at(0) || options[:host] + super(host) || (self[host] = Rfm::Server.new(*args)) #(host, options.rfm_filter(:account_name, :password, :delete=>true))) + # This part reconfigures the named server, if you pass it new config in the [] method. + # This breaks some specs in all [] methods in Factory. Consider undoing this. See readme-dev. + # super(host).config(options) if (options) + # super(host) + end + end # ServerFactory + + + class DbFactory < Rfm::CaseInsensitiveHash # :nodoc: all + # extend Config + # config :parent=>'@server' + def initialize(server) + extend Config + config :parent=>'@server' @server = server @loaded = false end - - def [](dbname) - super or (self[dbname] = Rfm::Database.new(dbname, @server)) + + def [](*args) + # was: (dbname, acnt=nil, pass=nil) + options = get_config(*args) + name = options[:strings].delete_at(0) || options[:database] + #account_name = options[:strings].delete_at(0) || options[:account_name] + #password = options[:strings].delete_at(0) || options[:password] + super(name) || (self[name] = Rfm::Database.new(@server, *args)) #(name, account_name, password, @server)) + # This part reconfigures the named database, if you pass it new config in the [] method. + # super(name).config({:account_name=>account_name, :password=>password}.merge(options)) if (account_name or password or options) + # super(name) end - + def all if !@loaded - Rfm::Result::ResultSet.new(@server, @server.connect(@server.state[:account_name], @server.state[:password], '-dbnames', {}).body).each {|record| - name = record['DATABASE_NAME'] - self[name] = Rfm::Database.new(name, @server) if self[name] == nil - } + c = Connection.new('-dbnames', {}, {:grammar=>'FMPXMLRESULT'}, @server) + c.parse('fmpxml_minimal.yml', {})['data'].each{|k,v| (self[k] = Rfm::Database.new(v['text'], @server)) if k.to_s != '' && v['text']} + #r = c.parse('fmpxml_minimal.yml', {}) @loaded = true end - self.values + self end - - end - - class LayoutFactory < Rfm::CaseInsensitiveHash - + + def names + self.values.collect{|v| v.name} + end + + end # DbFactory + + + + class LayoutFactory < Rfm::CaseInsensitiveHash # :nodoc: all + + # extend Config + # config :parent=>'@database' + def initialize(server, database) + extend Config + config :parent=>'@database' @server = server @database = database @loaded = false end - - def [](layout_name) - super or (self[layout_name] = Rfm::Layout.new(layout_name, @database)) + + def [](*args) # was layout_name + options = get_config(*args) + name = options[:strings].delete_at(0) || options[:layout] + super(name) || (self[name] = Rfm::Layout.new(@database, *args)) #(name, @database, options)) + # This part reconfigures the named layout, if you pass it new config in the [] method. + # super(name).config({:layout=>name}.merge(options)) if options + # super(name) end - + def all if !@loaded - Rfm::Result::ResultSet.new(@server, @server.connect(@server.state[:account_name], @server.state[:password], '-layoutnames', {"-db" => @database.name}).body).each {|record| - name = record['LAYOUT_NAME'] - self[name] = Rfm::Layout.new(name, @database) if self[name] == nil - } + c = Connection.new('-layoutnames', {"-db" => @database.name}, {:grammar=>'FMPXMLRESULT'}, @database) + c.parse('fmpxml_minimal.yml', {})['data'].each{|k,v| (self[k] = Rfm::Layout.new(v['text'], @database)) if k.to_s != '' && v['text']} @loaded = true end - self.values + self + end + + def names + values.collect{|v| v.name} + end + + # Acquired from Rfm::Base + def modelize(filter = /.*/) + all.values.each{|lay| lay.modelize if lay.name.match(filter)} + models + end + + # Acquired from Rfm::Base + def models + rslt = {} + each do |k,lay| + layout_models = lay.models + rslt[k] = layout_models if (!layout_models.nil? && !layout_models.empty?) + end + rslt end - - end - - class ScriptFactory < Rfm::CaseInsensitiveHash - + + end # LayoutFactory + + + + class ScriptFactory < Rfm::CaseInsensitiveHash # :nodoc: all + + # extend Config + # config :parent=>'@database' + def initialize(server, database) + extend Config + config :parent=>'@database' @server = server @database = database @loaded = false end - + def [](script_name) - super or (self[script_name] = Rfm::Script.new(script_name, @database)) + super or (self[script_name] = Rfm::Metadata::Script.new(script_name, @database)) end - + def all if !@loaded - Rfm::Result::ResultSet.new(@server, @server.connect(@server.state[:account_name], @server.state[:password], '-scriptnames', {"-db" => @database.name}).body).each {|record| - name = record['SCRIPT_NAME'] - self[name] = Rfm::Script.new(name, @database) if self[name] == nil - } + c = Connection.new('-scriptnames', {"-db" => @database.name}, {:grammar=>'FMPXMLRESULT'}, @database) + c.parse('fmpxml_minimal.yml', {})['data'].each{|k,v| (self[k] = Rfm::Metadata::Script.new(v['text'], @database)) if k.to_s != '' && v['text']} @loaded = true end - self.values + self + end + + def names + values.collect{|v| v.name} + end + + end # ScriptFactory + + + + class << self + + # Acquired from Rfm::Base + attr_accessor :models + # Shortcut to Factory.db().layouts.modelize() + # If first parameter is regex, it is used for modelize filter. + # Otherwise, parameters are passed to Factory.database + def modelize(*args) + regx = args[0].is_a?(Regexp) ? args.shift : /.*/ + db(*args).layouts.modelize(regx) end - - end - end -end \ No newline at end of file + + def servers + @servers ||= ServerFactory.new + end + + # Returns Rfm::Server instance, given config hash or array + def server(*conf) + Server.new(*conf) + end + + # Returns Rfm::Db instance, given config hash or array + def db(*conf) + Database.new(*conf) + end + + alias_method :database, :db + + # Returns Rfm::Layout instance, given config hash or array + def layout(*conf) + Layout.new(*conf) + end + + end # class << self + + end # Factory +end # Rfm diff --git a/lib/rfm/utilities/fmpxmlresult.xml.builder b/lib/rfm/utilities/fmpxmlresult.xml.builder new file mode 100644 index 00000000..89ee0cd5 --- /dev/null +++ b/lib/rfm/utilities/fmpxmlresult.xml.builder @@ -0,0 +1,33 @@ +#!/usr/bin/env ruby + +# Return @records as fmpxmlresult.xml +# Note that all element and attribute names in fmpxmlresult grammar MUST BE ALLCAPS. +@keys = @records[0].attribute_names +xml.FMPXMLRESULT :XMLNS=>"http://www.filemaker.com/fmpxmlresult" do + xml.ERRORCODE '0' + xml.PRODUCT :BUILD=>'1.0', :NAME=>'esalen', :VERSION=>'2' + xml.DATABASE :DATEFORMAT=>"M/d/yyyy", :LAYOUT=>'', :NAME=>'order_items', :RECORDS=>@records.size, :TIMEFORMAT=>"h:mm:ss a", :TIMESTAMPFORMAT=>"M/d/yyyy h:mm:ss a" + xml.METADATA do + @keys.each do |name| + type = case @records[0][name].class.to_s + when /decimal|fixnum|int|float/i; "NUMBER" + when /^time$/i; "TIMESTAMP" # All sql timestamp fields seem to be coming thru as "Time". + when /^date$/i; "DATE" + when /timestamp|datetime/i; "TIMESTAMP" + else "TEXT" + end + xml.FIELD :NAME=>name, :EMPTYOK=>'yes', :MAXREPEAT=>"1", :TYPE=>type + end + end + xml.RESULTSET(:FOUND=>@records.size.to_i) do + @records.each do |r| + xml.ROW :MODID=>"0", :RECORDID=>r.id do + @keys.each do |k| + xml.COL do + xml.DATA r[k] + end + end + end + end + end +end diff --git a/lib/rfm/utilities/sax/fmpxml_minimal.yml b/lib/rfm/utilities/sax/fmpxml_minimal.yml new file mode 100644 index 00000000..1c2a448d --- /dev/null +++ b/lib/rfm/utilities/sax/fmpxml_minimal.yml @@ -0,0 +1,47 @@ +#!/usr/bin/env ruby +# Database names from fmpxmlresult # +# +# +# +# +# 0 +# +# +# +# +# +# +# +# +# SDS +# +# +# +# +# SevenDev +# +# +# +# +# SevenGables +# +# +# +# +# +# The top level represents the initial object, a basic hash in this case. +--- +attach_elements: none +attach_attributes: none +elements: +- name: data + compact: true + attach: hash + attach_attributes: hash + delimiter: text +# Experimental, but needs to be done. +# - name: errorcode +# as_name: error +# attach: private +# compact: true + diff --git a/lib/rfm/utilities/sax/fmpxmllayout.yml b/lib/rfm/utilities/sax/fmpxmllayout.yml new file mode 100644 index 00000000..22647ccd --- /dev/null +++ b/lib/rfm/utilities/sax/fmpxmllayout.yml @@ -0,0 +1,190 @@ +#!/usr/bin/env ruby +# YAML structure defining a SAX parsing scheme for fmpxmllayout xml. +# The initial object of this parse should be a new instance of Rfm::Layout. +--- +attach_elements: _meta +attach_attributes: _meta +elements: +- name: doctype + attach: none + attributes: + - name: value + as_name: doctype +- name: fmpxmllayout + attach: none +- name: errorcode + attach: none + before_close: :check_for_errors + attributes: + - name: text + as_name: error +- name: product + as_name: product +- name: layout + attach: none + attributes: + - name: database + as_name: database + - name: NAME + as_name: name +- name: field + attach: + - cursor + - Rfm::Metadata::FieldControl + - :new + - ivg(:meta) + # Must use before_close handler to attach, since field_mapping must be applied to value-list key name. + before_close: [object, ':element_close_handler'] + # # Used to be 'object.meta', but that required sloppy 'loaded' indicator handling (or infinite loop), + # # so now just referring to raw inst var @meta, instead of method .meta. + # # Need to assume attributes may or may not be included in start_element call. + # handler: [object.instance_variable_get('@meta'), handle_new_field_control, _attributes] + # # attach_attributes isn't doing anything when used with new-element-handler. + attach_attributes: private + delimiter: name + elements: + # This config doesn't use the creation handler, so it should work with ox. + - name: style + #element_handler: [object, handle_style_element, _attributes] + attach: none + #attach: [cursor, object, handle_style_element, _attributes] + #attach_attributes: none + attributes: + - name: valuelist + as_name: value_list_name + #translate: translate_style_value +- name: valuelists + attach: none +- name: valuelist + #class: Array + attach: [_meta, Array, ':new'] + as_name: value_lists + delimiter: name + elements: + - name: value + #class: Rfm::Metadata::ValueListItem + attach: [array, 'Rfm::Metadata::ValueListItem', ':allocate'] + before_close: replace(@value.to_s) + attach_attributes: private + attributes: + - name: display + as_name: display + - name: text + as_name: value + + +# Works - doesn't use Handler method. +# --- +# attach_elements: _meta +# attach_attributes: _meta +# elements: +# - name: doctype +# attach: none +# attributes: +# - name: value +# as_name: doctype +# - name: FMPXMLLAYOUT +# attach: none +# - name: ERRORCODE +# attach: none +# # TODO: Build check_for_errors method in Layout class. +# #before_close: 'check_for_errors' +# attributes: +# - name: text +# as_name: error +# - name: PRODUCT +# as_name: product +# - name: LAYOUT +# attach: none +# attributes: +# - name: DATABASE +# as_name: database +# - name: NAME +# as_name: name +# - name: FIELD +# class: Rfm::Metadata::FieldControl +# as_name: field_controls +# delimiter: name +# attach_attributes: private +# attributes: +# - name: NAME +# as_name: name +# elements: +# - name: STYLE +# attach: none +# # Add logic for 3rd level of prefs, so FieldControl pref captures this attach_attributes instead having to explicitly define here. +# attach_attributes: private +# attributes: +# - name: TYPE +# as_name: type +# - name: VALUELIST +# as_name: valuelist +# - name: VALUELISTS +# attach: none +# - name: VALUELIST +# # TODO: Disabling this breaks the delimiter. +# #attach: shared +# class: Array +# as_name: value_lists +# delimiter: NAME +# elements: +# - name: VALUE +# class: Rfm::Metadata::ValueListItem +# before_close: replace(@value.to_s) +# attach_attributes: private +# attributes: +# - name: DISPLAY +# as_name: display +# - name: text +# as_name: value + + + +# Works !!! +# --- +# elements: +# - name: FMPXMLLAYOUT +# attach: none +# attach_attributes: shared +# - name: ERRORCODE +# as_name: error +# attach: shared +# compact: true +# - name: PRODUCT +# attach: none +# attach_attributes: shared +# - name: LAYOUT +# attach: none +# attach_attributes: shared +# - name: FIELD +# class: Rfm::Metadata::FieldControl +# as_name: field_controls +# delineate_with_hash: name +# attributes: +# - name: NAME +# as_name: name +# elements: +# - name: STYLE +# attach: none +# attributes: +# - name: TYPE +# as_name: type +# - name: VALUELIST +# as_name: valuelist +# - name: VALUELISTS +# attach: none +# - name: VALUELIST +# class: Array +# as_name: value_lists +# delineate_with_hash: NAME +# elements: +# - name: VALUE +# # This turns the potential hash into a string +# class: String +# # Then we place the main attribute into the string's data. +# before_close: replace text +# attributes: +# - name: DISPLAY +# as_name: display + + diff --git a/lib/rfm/utilities/sax/fmresultset.xml b/lib/rfm/utilities/sax/fmresultset.xml new file mode 100644 index 00000000..48917c54 --- /dev/null +++ b/lib/rfm/utilities/sax/fmresultset.xml @@ -0,0 +1,79 @@ + + + individual + + + fmresultset + Rfm::FmResultset + + self + + + + datasource + Rfm::Datasource + + + metadata + Rfm::Meta + + + field_definition + field_meta + hash + + + portal_definition + portal_meta + hash + + + + + resultset + individual + + self + + + + record + Rfm::Record + individual + hash + + + field + Rfm::Metadata::Field + individual + true + none + build_record_data + + + relatedset + Rfm::RelatedSet + individual + portals + table + + + record + Rfm::Record + individual + + + field + true + Rfm::Metadata::Field + none + build_record_data + + + + + + + + + \ No newline at end of file diff --git a/lib/rfm/utilities/sax/fmresultset.yml b/lib/rfm/utilities/sax/fmresultset.yml new file mode 100644 index 00000000..e1062522 --- /dev/null +++ b/lib/rfm/utilities/sax/fmresultset.yml @@ -0,0 +1,166 @@ +#!/usr/bin/env ruby +# YAML structure defining a SAX parsing scheme for fmresultset xml. +# The initial object of this parse should be a new instance of Rfm::Resultset. +--- +attach_elements: _meta +attach_attributes: _meta +create_accessors: all +elements: +- name: doctype + attach: none + attributes: + - name: value + as_name: doctype +- name: fmresultset + attach: none +- name: product +- name: error + attach: none + before_close: :check_for_errors + attributes: + - name: code + as_name: error +- name: datasource + attach: none + before_close: [object, end_datasource_element_callback, self] + attributes: + - name: total_count + accessor: none +- name: metadata + attach: none +- name: field_definition + # These two steps can be used to create the attachment to resultset-meta automatically, + # but the field-mapping translation won't happen. + # attach: [_meta, 'Rfm::Metadata::Field', allocate] + # as_name: field_meta + attach: [cursor, 'Rfm::Metadata::Field', ':allocate'] + delimiter: name + attach_attributes: private + before_close: [object, field_definition_element_close_callback, self] +- name: relatedset_definition + delimiter: table + as_name: portal_meta + attach_attributes: private + elements: + - name: field_definition + attach: [cursor, 'Rfm::Metadata::Field', ':allocate'] + delimiter: name + as_name: field_meta + attach_attributes: private + before_close: [object, relatedset_field_definition_element_close_callback, self] +- name: resultset + attach: none + attributes: + - name: count + accessor: none + - name: fetch_size + accessor: none +- name: record + #attach: [cursor, object, handle_new_record, _attributes] + #attach_attributes: none + attach: [array, 'Rfm::Record', new, object] + attach_attributes: private + before_close: '@loaded=true' + elements: + - name: field + attach: [cursor, 'Rfm::Metadata::Datum', ':allocate'] + compact: false + before_close: [object, field_element_close_callback, self] + - name: relatedset + attach: [private, Array, ':allocate'] + as_name: portals + attach_attributes: private + create_accessors: all + delimiter: table + elements: + - name: record + #class: Rfm::Record + attach: [default, 'Rfm::Record', ':allocate'] + attach_attributes: private + before_close: '@loaded=true' + elements: + - name: field + compact: true + attach: [cursor, 'Rfm::Metadata::Datum', ':allocate'] + before_close: [object, portal_field_element_close_callback, self] + + + +# Works - doesn't use Handler methods. +# --- +# attach_elements: _meta +# attach_attributes: _meta +# create_accessors: all +# elements: +# - name: doctype +# attach: none +# attributes: +# - name: value +# as_name: doctype +# - name: fmresultset +# attach: none +# - name: product +# - name: error +# attach: none +# before_close: 'check_for_errors' +# attributes: +# - name: code +# as_name: error +# - name: datasource +# attach: none +# before_close: :end_datasource_element_callback +# - name: metadata +# attach: none +# - name: field_definition +# class: Rfm::Metadata::Field +# attach: cursor +# delimiter: name +# as_name: field_meta +# attach_attributes: private +# before_close: :main_callback +# - name: relatedset_definition +# delimiter: table +# as_name: portal_meta +# attach_attributes: private +# elements: +# - name: field_definition +# class: Rfm::Metadata::Field +# attach: cursor +# delimiter: name +# as_name: field_meta +# attach_attributes: private +# before_close: :portal_callback +# - name: resultset +# attach: none +# - name: record +# class: Rfm::Record +# initialize: [new, object] +# attach: array +# attach_attributes: private +# before_close: '@loaded=true' +# elements: +# - name: field +# class: Rfm::Metadata::Datum +# compact: true +# attach: cursor +# before_close: :main_callback +# - name: relatedset +# class: Array +# as_name: portals +# attach: private +# attach_attributes: private +# create_accessors: all +# delimiter: table +# elements: +# - name: record +# class: Rfm::Record +# # Should this be enabled? +# #initialize: [new, object] +# attach_attributes: private +# before_close: '@loaded=true' +# elements: +# - name: field +# compact: true +# class: Rfm::Metadata::Datum +# attach: cursor +# before_close: :portal_callback diff --git a/lib/rfm/utilities/sax_parser.rb b/lib/rfm/utilities/sax_parser.rb new file mode 100644 index 00000000..5977a830 --- /dev/null +++ b/lib/rfm/utilities/sax_parser.rb @@ -0,0 +1,1229 @@ +# encoding: UTF-8 +# Encoding is necessary for Ox, which appears to ignore character encoding. +# See: http://stackoverflow.com/questions/11331060/international-chars-using-rspec-with-ruby-on-rails +# +# #### A declarative SAX parser, written by William Richardson ##### +# +# This XML parser builds a result object from callbacks sent by any ruby sax/stream parsing +# engine. The engine can be any ruby xml parser that offers a sax or streaming parsing scheme +# that sends callbacks to a handler object for the various node events encountered in the xml stream. +# The currently supported parsers are the Ruby librarys libxml-ruby, nokogiri, ox, and rexml. +# +# Without a configuration template, this parser will return a generic tree of hashes and arrays, +# representing the xml structure & data that was fed into the parser. With a configuration template, +# this parser can create resulting objects that are custom transformations of the input xml. +# +# The goal in writing this parser was to build custom objects from xml, in a single pass, +# without having to build a generic tree first and then pick it apart with ugly code scattered all over +# our projects' classes. +# +# The primaray use case that motivated this parser's construction was converting Filemaker Server's +# xml response documents into Ruby result-set arrays containing record hashes. A primary example of +# this use can be seen in the Ruby gem 'ginjo-rfm' (a Ruby-Filemaker adapter). +# +# +# Useage: +# irb -rubygems -I./ -r lib/rfm/utilities/sax_parser.rb +# SaxParser.parse(io, template=nil, initial_object=nil, parser=nil, options={}) +# io: xml-string or xml-file-path or file-io or string-io +# template: file-name, yaml, xml, symbol, or hash +# initial_object: the parent object - any object to which the resulting build will be attached to. +# parser: backend parser symbol or custom backend handler instance +# options: extra options +# +# +# Note: 'attach: cursor' puts the object in the cursor & stack but does not attach it to the parent. +# 'attach: none' prevents the object from entering the cursor or stack. +# Both of these will still allow processing of attributes and subelements. +# +# Note: Attribute attachment is controlled first by the attributes' model's :attributes hash (controls individual attrs), +# and second by the base model's main hash. Any model's main hash :attach_attributes only controls +# attributes that will be attached to that model's object. So if a model's object is not attached to anything +# (:attach=>'none'), then the higher base-model's :attach_attributes will control the lower model's attribute attachment. +# Put another way: If a model is :attach=>'none', then its :attach_attributes won't be counted. +# +# +# Examples: +# Rfm::SaxParser.parse('some/file.xml') # => defaults to best xml backend with no parsing configuration. +# Rfm::SaxParser.parse('some/file.xml', nil, nil, :ox) # => uses ox backend or throws error. +# Rfm::SaxParser.parse('some/file.xml', {'compact'=>true}, :rexml) # => uses inline configuration with rexml parser. +# Rfm::SaxParser.parse('some/file.xml', 'path/to/config.yml', SomeClass.new) # => loads config template from yml file and builds on top of instance of SomeClass. +# +# +# #### CONFIGURATION ##### +# +# YAML structure defining a SAX xml parsing template. +# An element may contain these config directives. +# An attribute may contain some of these config directives. +# Options: +# initialize_with: OBSOLETE? string, symbol, or array (object, method, params...). Should return new object. See Rfm::SaxParser::Cursor#get_callback. +# elements: array of element hashes [{'name'=>'element-tag'},...] +# attributes: array of attribute hashes {'name'=>'attribute-name'} UC +# class: string-or-class: class name for new element +# attach: string: shared, _shared_var_name, private, hash, array, cursor, none - how to attach this element or attribute to #object. +# array: [0]string of above, [1..-1]new_element_callback options (see get_callback method). +# attach_elements: string: same as 'attach' - how to attach ANY subelements to this model's object, unless they have their own 'attach' specification. +# attach_attributes: string: same as 'attach' - how to attach ANY attributes to this model's object, unless they have their own 'attach' specification. +# before_close: string, symbol, or array (object, method, params...). See Rfm::SaxParser::Cursor#get_callback. +# as_name: string: store element or attribute keyed as specified +# delimiter: string: attribute/hash key to delineate objects with identical tags +# create_accessors: string or array: all, private, shared, hash, none +# accessor: string: all, private, shared, hash, none +# element_handler: NOT-USED? string, symbol, or array (object, method, params...). Should return new object. See Rfm::SaxParser::Cursor#get_callback. +# Default attach prefs are 'cursor'. +# Use this when all new-element operations should be offloaded to custom class or module. +# Should return an instance of new object. +# translate: UC Consider adding a 'translate' option to point to a method on the current model's object to use to translate values for attributes. +# +# +# #### See below for notes & todos #### + + +require 'yaml' +require 'forwardable' +require 'stringio' + +module Rfm + module SaxParser + + RUBY_VERSION_NUM = RUBY_VERSION[0,3].to_f + + PARSERS = {} + + # These defaults can be set here or in any ancestor/enclosing module or class, + # as long as the defaults or their constants can be seen from this POV. + # + # Default class MUST be a descendant of Hash or respond to hash methods !!! + # + # For backend, use :libxml, :nokogiri, :ox, :rexml, or anything else, if you want it to always default + # to something other than the fastest backend found. + # Using nil will let the SaxParser decide. + @parser_defaults = { + :default_class => Hash, + :backend => nil, + :text_label => 'text', + :tag_translation => lambda {|txt| txt.gsub(/\-/, '_').downcase}, + :shared_variable_name => 'attributes', + :templates => {}, + :template_prefix => nil + } + + # Merge any upper-level default definitions + if defined? PARSER_DEFAULTS + tmp_defaults = PARSER_DEFAULTS.dup + PARSER_DEFAULTS.replace(@parser_defaults).merge!(tmp_defaults) + else + PARSER_DEFAULTS = @parser_defaults + end + + # Convert defaults to constants, available to all sub classes/modules/instances. + PARSER_DEFAULTS.each do |k, v| + k = k.to_s.upcase + #(const_set k, v) unless eval("defined? #{k}") #(const_defined?(k) or defined?(k)) + if eval("defined? #{k}") + (const_set k, eval(k)) + else + (const_set k, v) + end + end + + ::Object::ATTACH_OBJECT_DEFAULT_OPTIONS = { + :shared_variable_name => SHARED_VARIABLE_NAME, + :default_class => DEFAULT_CLASS, + :text_label => TEXT_LABEL, + :create_accessors => [] #:all, :private, :shared, :hash + } + + def self.parse(*args) + Handler.build(*args) + end + + # A Cursor instance is created for each element encountered in the parsing run + # and is where the parsing result is constructed from the custom parsing template. + # The cursor is the glue between the handler and the resulting object build. The + # cursor receives input from the handler, loads the corresponding template data, + # and manipulates the incoming xml data to build the resulting object. + # + # Each cursor is added to the stack when its element begins, and removed from + # the stack when its element ends. The cursor tracks all the important objects + # necessary to build the resulting object. If you call #cursor on the handler, + # you will always get the last object added to the stack. Think of a cursor as + # a framework of tools that accompany each element's build process. + class Cursor + extend Forwardable + + # model - currently active model (rename to current_model) + # local_model - model of this cursor's tag (rename to local_model) + # newtag - incoming tag of yet-to-be-created cursor. Get rid of this if you can. + # element_attachment_prefs - local object's attachment prefs based on local_model and current_model. + # level - cursor depth + attr_accessor :model, :local_model, :object, :tag, :handler, :parent, :level, :element_attachment_prefs, :new_element_callback, :initial_attributes #, :newtag + + + #SaxParser.install_defaults(self) + + def_delegators :handler, :top, :stack + + # Main get-constant method + def self.get_constant(klass) + #puts "Getting constant '#{klass.to_s}'" + + case + when klass.is_a?(Class); klass + #when (klass=klass.to_s) == ''; DEFAULT_CLASS + when klass.nil?; DEFAULT_CLASS + when klass == ''; DEFAULT_CLASS + when klass[/::/]; eval(klass) + when defined?(klass); const_get(klass) ## == 'constant'; const_get(klass) + #when defined?(klass); eval(klass) # This was for 'element_handler' pattern. + else + Rfm.log.warn "Could not find constant '#{klass}'" + DEFAULT_CLASS + end + end + + def initialize(_tag, _handler, _parent=nil, _initial_attributes=nil) #, caller_binding=nil) + #def initialize(_model, _obj, _tag, _handler) + @tag = _tag + @handler = _handler + @parent = _parent || self + @initial_attributes = _initial_attributes + @level = @parent.level.to_i + 1 + @local_model = (model_elements?(@tag, @parent.model) || DEFAULT_CLASS.new) + @element_attachment_prefs = attachment_prefs(@parent.model, @local_model, 'element') + #@attribute_attachment_prefs = attachment_prefs(@parent.model, @local_model, 'attribute') + + if @element_attachment_prefs.is_a? Array + @new_element_callback = @element_attachment_prefs[1..-1] + @element_attachment_prefs = @element_attachment_prefs[0] + if @element_attachment_prefs.to_s == 'default' + @element_attachment_prefs = nil + end + end + + #puts ["\nINITIALIZE_CURSOR tag: #{@tag}", "parent.object: #{@parent.object.class}", "local_model: #{@local_model.class}", "el_prefs: #{@element_attachment_prefs}", "new_el_callback: #{@new_element_callback}", "attributes: #{@initial_attributes}"] + + self + end + + + ##### SAX METHODS ##### + + # Receive a single attribute (any named attribute or text) + def receive_attribute(name, value) + #puts ["\nRECEIVE_ATTR '#{name}'", "value: #{value}", "tag: #{@tag}", "object: #{object.class}", "model: #{model['name']}"] + new_att = {name=>value} #.new.tap{|att| att[name]=value} + assign_attributes(new_att) #, @object, @model, @local_model) + rescue + Rfm.log.warn "Error: could not assign attribute '#{name.to_s}' to element '#{self.tag.to_s}': #{$!}" + end + + def receive_start_element(_tag, _attributes) + #puts ["\nRECEIVE_START '#{_tag}'", "current_object: #{@object.class}", "current_model: #{@model['name']}", "attributes #{_attributes}"] + new_cursor = Cursor.new(_tag, @handler, self, _attributes) #, binding) + new_cursor.process_new_element(binding) + new_cursor + end # receive_start_element + + # Decides how to attach element & attributes associated with this cursor. + def process_new_element(caller_binding=binding) + #puts ["\nPROCESS_NEW_ELEMENT tag: #{@tag}", "@element_attachment_prefs: #{@element_attachment_prefs}", "@local_model: #{local_model}"] + + new_element = @new_element_callback ? get_callback(@new_element_callback, caller_binding) : nil + + case + # when inital cursor, just set model & object. + when @tag == '__TOP__'; + #puts "__TOP__" + @model = @handler.template + @object = @handler.initial_object + + when @element_attachment_prefs == 'none'; + #puts "__NONE__" + @model = @parent.model #nil + @object = @parent.object #nil + + if @initial_attributes && @initial_attributes.any? #&& @attribute_attachment_prefs != 'none' + assign_attributes(@initial_attributes) #, @object, @model, @local_model) + end + + when @element_attachment_prefs == 'cursor'; + #puts "__CURSOR__" + @model = @local_model + @object = new_element || DEFAULT_CLASS.allocate + + if @initial_attributes && @initial_attributes.any? #&& @attribute_attachment_prefs != 'none' + assign_attributes(@initial_attributes) #, @object, @model, @local_model) + end + + else + #puts "__OTHER__" + @model = @local_model + @object = new_element || DEFAULT_CLASS.allocate + + if @initial_attributes && @initial_attributes.any? #&& @attribute_attachment_prefs != 'none' + #puts "PROCESS_NEW_ELEMENT calling assign_attributes with ATTRIBUTES #{@initial_attributes}" + assign_attributes(@initial_attributes) #, @object, @model, @local_model) + end + + # If @local_model has a delimiter, defer attach_new_element until later. + #puts "PROCESS_NEW_ELEMENT delimiter of @local_model #{delimiter?(@local_model)}" + if !delimiter?(@local_model) + #attach_new_object(@parent.object, @object, @tag, @parent.model, @local_model, 'element') + #puts "PROCESS_NEW_ELEMENT calling attach_new_element with TAG #{@tag} and OBJECT #{@object}" + attach_new_element(@tag, @object) + end + end + + self + end + + + def receive_end_element(_tag) + #puts ["\nRECEIVE_END_ELEMENT '#{_tag}'", "tag: #{@tag}", "object: #{@object.class}", "model: #{@model['name']}", "local_model: #{@local_model['name']}"] + #puts ["\nEND_ELEMENT_OBJECT", object.to_yaml] + begin + + if _tag == @tag && (@model == @local_model) + # Data cleaup + compactor_settings = compact? || compact?(top.model) + #(compactor_settings = compact?(top.model)) unless compactor_settings # prefer local settings, or use top settings. + (clean_members {|v| clean_members(v){|w| clean_members(w)}}) if compactor_settings + end + + if (delimiter = delimiter?(@local_model); delimiter && !['none','cursor'].include?(@element_attachment_prefs.to_s)) + #attach_new_object(@parent.object, @object, @tag, @parent.model, @local_model, 'element') + #puts "RECEIVE_END_ELEMENT attaching new element TAG (#{@tag}) OBJECT (#{@object.class}) #{@object.to_yaml} WITH LOCAL MODEL #{@local_model.to_yaml} TO PARENT (#{@parent.object.class}) #{@parent.object.to_yaml} PARENT MODEL #{@parent.model.to_yaml}" + attach_new_element(@tag, @object) + end + + if _tag == @tag #&& (@model == @local_model) + # End-element callbacks. + #run_callback(_tag, self) + callback = before_close?(@local_model) + get_callback(callback, binding) if callback + end + + if _tag == @tag + # return true only if matching tags + return true + end + + # # return true only if matching tags + # if _tag == @tag + # return true + # end + + return + # rescue + # Rfm.log.debug "Error: end_element tag '#{_tag}' failed: #{$!}" + end + end + + ### Parse callback instructions, compile & send callback method ### + ### TODO: This is way too convoluted. Document it better, or refactor!!! + # This method will send a method to an object, with parameters, and return a new object. + # Input (first param): string, symbol, or array of strings + # Returns: object + # Default options: + # :object=>object + # :method=>'a method name string or symbol' + # :params=>"params string to be eval'd in context of cursor" + # Usage: + # callback: send a method (or eval string) to an object with parameters, consisting of... + # string: a string to be eval'd in context of current object. + # or symbol: method to be called on current object. + # or array: object, method, params. + # object: + # method: + # params: + # + # TODO-MAYBE: Change param order to (method, object, params), + # might help confusion with param complexities. + # + def get_callback(callback, caller_binding=binding, defaults={}) + input = callback.is_a?(Array) ? callback.dup : callback + #puts "\nGET_CALLBACK tag: #{tag}, callback: #{callback}" + params = case + when input.is_a?(String) || input.is_a?(Symbol) + [nil, input] + # when input.is_a?(Symbol) + # [nil, input] + when input.is_a?(Array) + #puts ["\nCURSOR#get_callback is an array", input] + case + when input[0].is_a?(Symbol) + [nil, input].flatten(1) + when input[1].is_a?(String) && ( input.size > 2 || (remove_colon=(input[1][0,1]==":"); remove_colon) ) + code_or_method = input[1].dup + code_or_method[0]='' if remove_colon + code_or_method = code_or_method.to_sym + output = [input[0], code_or_method, input[2..-1]].flatten(1) + #puts ["\nCURSOR#get_callback converted input[1] to symbol", output] + output + else # when input is ['object', 'sym-or-str', 'param1',' param2', ...] + input + end + else + [] + end + + obj_raw = params.shift + #puts ["\nOBJECT_RAW:","class: #{obj_raw.class}", "object: #{obj_raw}"] + obj = if obj_raw.is_a?(String) + eval(obj_raw.to_s, caller_binding) + else + obj_raw + end + if obj.nil? || obj == '' + obj = defaults[:object] || @object + end + #puts ["\nOBJECT:","class: #{obj.class}", "object: #{obj}"] + + code = params.shift || defaults[:method] + params.each_with_index do |str, i| + if str.is_a?(String) + params[i] = eval(str, caller_binding) + end + end + params = defaults[:params] if params.size == 0 + #puts ["\nGET_CALLBACK tag: #{@tag}" ,"callback: #{callback}", "obj.class: #{obj.class}", "code: #{code}", "params-class #{params.class}"] + case + when (code.nil? || code=='') + obj + when (code.is_a?(Symbol) || params) + #puts ["\nGET_CALLBACK sending symbol", obj.class, code] + obj.send(*[code, params].flatten(1).compact) + when code.is_a?(String) + #puts ["\nGET_CALLBACK evaling string", obj.class, code] + obj.send :eval, code + #eval(code, caller_binding) + end + end + + # # Run before-close callback. + # def run_callback(_tag, _cursor=self, _model=_cursor.local_model, _object=_cursor.object ) + # callback = before_close?(_model) + # #puts ["\nRUN_CALLBACK", _tag, _cursor.tag, _object.class, callback, callback.class] + # if callback.is_a? Symbol + # _object.send callback, _cursor + # elsif callback.is_a?(String) + # _object.send :eval, callback + # end + # end + + + + + ##### MERGE METHODS ##### + + # Assign attributes to element. + def assign_attributes(_attributes) + if _attributes && !_attributes.empty? + + _attributes.each do |k,v| + #attach_new_object(base_object, v, k, base_model, model_attributes?(k, new_model), 'attribute')} + attr_model = model_attributes?(k, @local_model) + + label = label_or_tag(k, attr_model) + + prefs = [attachment_prefs(@model, attr_model, 'attribute')].flatten(1)[0] + + shared_var_name = shared_variable_name(prefs) + (prefs = "shared") if shared_var_name + + # Use local create_accessors prefs first, then more general ones. + create_accessors = accessor?(attr_model) || create_accessors?(@model) + #(create_accessors = create_accessors?(@model)) unless create_accessors && create_accessors.any? + + #puts ["\nATTACH_NEW_OBJECT 1", "type: #{type}", "label: #{label}", "base_object: #{base_object.class}", "new_object: #{new_object.class}", "delimiter: #{delimiter?(new_model)}", "prefs: #{prefs}", "shared_var_name: #{shared_var_name}", "create_accessors: #{create_accessors}"] + @object._attach_object!(v, label, delimiter?(attr_model), prefs, 'attribute', :default_class=>DEFAULT_CLASS, :shared_variable_name=>shared_var_name, :create_accessors=>create_accessors) + end + + end + end + + # def attach_new_object(base_object, new_object, name, base_model, new_model, type) + # label = label_or_tag(name, new_model) + # + # # Was this, which works fine, but not as efficient: + # # prefs = [attachment_prefs(base_model, new_model, type)].flatten(1)[0] + # prefs = if type=='attribute' + # [attachment_prefs(base_model, new_model, type)].flatten(1)[0] + # else + # @element_attachment_prefs + # end + # + # shared_var_name = shared_variable_name(prefs) + # (prefs = "shared") if shared_var_name + # + # # Use local create_accessors prefs first, then more general ones. + # create_accessors = accessor?(new_model) + # (create_accessors = create_accessors?(base_model)) unless create_accessors && create_accessors.any? + # + # # # This is NEW! + # # translator = new_model['translator'] + # # if translator + # # new_object = base_object.send translator, name, new_object + # # end + # + # + # #puts ["\nATTACH_NEW_OBJECT 1", "type: #{type}", "label: #{label}", "base_object: #{base_object.class}", "new_object: #{new_object.class}", "delimiter: #{delimiter?(new_model)}", "prefs: #{prefs}", "shared_var_name: #{shared_var_name}", "create_accessors: #{create_accessors}"] + # base_object._attach_object!(new_object, label, delimiter?(new_model), prefs, type, :default_class=>DEFAULT_CLASS, :shared_variable_name=>shared_var_name, :create_accessors=>create_accessors) + # #puts ["\nATTACH_NEW_OBJECT 2: #{base_object.class} with ", label, delimiter?(new_model), prefs, type, :shared_variable_name=>shared_var_name, :create_accessors=>create_accessors] + # # if type == 'attribute' + # # puts ["\nATTACH_ATTR", "name: #{name}", "label: #{label}", "new_object: #{new_object.class rescue ''}", "base_object: #{base_object.class rescue ''}", "base_model: #{base_model['name'] rescue ''}", "new_model: #{new_model['name'] rescue ''}", "prefs: #{prefs}"] + # # end + # end + + def attach_new_element(name, new_object) #old params (base_object, new_object, name, base_model, new_model, type) + label = label_or_tag(name, @local_model) + + # Was this, which works fine, but not as efficient: + # prefs = [attachment_prefs(base_model, new_model, type)].flatten(1)[0] + prefs = @element_attachment_prefs + + shared_var_name = shared_variable_name(prefs) + (prefs = "shared") if shared_var_name + + # Use local create_accessors prefs first, then more general ones. + create_accessors = accessor?(@local_model) || create_accessors?(@parent.model) + #(create_accessors = create_accessors?(@parent.model)) unless create_accessors && create_accessors.any? + + # # This is NEW! + # translator = new_model['translator'] + # if translator + # new_object = base_object.send translator, name, new_object + # end + + + #puts ["\nATTACH_NEW_ELEMENT 1", "new_object: #{new_object}", "parent_object: #{@parent.object}", "label: #{label}", "delimiter: #{delimiter?(@local_model)}", "prefs: #{prefs}", "shared_var_name: #{shared_var_name}", "create_accessors: #{create_accessors}"] + @parent.object._attach_object!(new_object, label, delimiter?(@local_model), prefs, 'element', :default_class=>DEFAULT_CLASS, :shared_variable_name=>shared_var_name, :create_accessors=>create_accessors) + # if type == 'attribute' + # puts ["\nATTACH_ATTR", "name: #{name}", "label: #{label}", "new_object: #{new_object.class rescue ''}", "base_object: #{base_object.class rescue ''}", "base_model: #{base_model['name'] rescue ''}", "new_model: #{new_model['name'] rescue ''}", "prefs: #{prefs}"] + # end + end + + def attachment_prefs(base_model, new_model, type) + case type + when 'element'; attach?(new_model) || attach_elements?(base_model) #|| attach?(top.model) || attach_elements?(top.model) + when 'attribute'; attach?(new_model) || attach_attributes?(base_model) #|| attach?(top.model) || attach_attributes?(top.model) + end + end + + def shared_variable_name(prefs) + rslt = nil + if prefs.to_s[0,1] == "_" + rslt = prefs.to_s[1..-1] #prefs.gsub(/^_/, '') + end + end + + + ##### UTILITY ##### + + def get_constant(klass) + self.class.get_constant(klass) + end + + # Methods for current _model + def ivg(name, _object=@object); _object.instance_variable_get "@#{name}"; end + def ivs(name, value, _object=@object); _object.instance_variable_set "@#{name}", value; end + def model_elements?(which=nil, _model=@model); _model && _model.has_key?('elements') && ((_model['elements'] && which) ? _model['elements'].find{|e| e['name']==which} : _model['elements']) ; end + def model_attributes?(which=nil, _model=@model); _model && _model.has_key?('attributes') && ((_model['attributes'] && which) ? _model['attributes'].find{|a| a['name']==which} : _model['attributes']) ; end + def depth?(_model=@model); _model && _model['depth']; end + def before_close?(_model=@model); _model && _model['before_close']; end + def each_before_close?(_model=@model); _model && _model['each_before_close']; end + def compact?(_model=@model); _model && _model['compact']; end + def attach?(_model=@model); _model && _model['attach']; end + def attach_elements?(_model=@model); _model && _model['attach_elements']; end + def attach_attributes?(_model=@model); _model && _model['attach_attributes']; end + def delimiter?(_model=@model); _model && _model['delimiter']; end + def as_name?(_model=@model); _model && _model['as_name']; end + def initialize_with?(_model=@model); _model && _model['initialize_with']; end + def create_accessors?(_model=@model); _model && _model['create_accessors'] && [_model['create_accessors']].flatten.compact; end + def accessor?(_model=@model); _model && _model['accessor'] && [_model['accessor']].flatten.compact; end + def element_handler?(_model=@model); _model && _model['element_handler']; end + + + # Methods for submodel + + # This might be broken. + def label_or_tag(_tag=@tag, new_model=@local_model); as_name?(new_model) || _tag; end + + + def clean_members(obj=@object) + #puts ["CURSOR.clean_members: #{object.class}", "tag: #{tag}", "model-name: #{model[:name]}"] + # cursor.object = clean_member(cursor.object) + # clean_members(ivg(shared_attribute_var, obj)) + if obj.is_a?(Hash) + obj.dup.each do |k,v| + obj[k] = clean_member(v) + yield(v) if block_given? + end + elsif obj.is_a?(Array) + obj.dup.each_with_index do |v,i| + obj[i] = clean_member(v) + yield(v) if block_given? + end + else + obj.instance_variables.each do |var| + dat = obj.instance_variable_get(var) + obj.instance_variable_set(var, clean_member(dat)) + yield(dat) if block_given? + end + end + # obj.instance_variables.each do |var| + # dat = obj.instance_variable_get(var) + # obj.instance_variable_set(var, clean_member(dat)) + # yield(dat) if block_given? + # end + end + + def clean_member(val) + if val.is_a?(Hash) || val.is_a?(Array); + if val && val.empty? + nil + elsif val && val.respond_to?(:values) && val.size == 1 + val.values[0] + else + val + end + else + val + # # Probably shouldn't do this on instance-var values. ...Why not? + # if val.instance_variables.size < 1 + # nil + # elsif val.instance_variables.size == 1 + # val.instance_variable_get(val.instance_variables[0]) + # else + # val + # end + end + end + + end # Cursor + + + + ##### SAX HANDLER ##### + + + # A handler instance is created for each parsing run. The handler has several important functions: + # 1. Receive callbacks from the sax/stream parsing engine (start_element, end_element, attribute...). + # 2. Maintain a stack of cursors, growing & shrinking, throughout the parsing run. + # 3. Maintain a Cursor instance throughout the parsing run. + # 3. Hand over parser callbacks & data to the Cursor instance for refined processing. + # + # The handler instance is unique to each different parsing gem but inherits generic + # methods from this Handler module. During each parsing run, the Hander module creates + # a new instance of the spcified parer's handler class and runs the handler's main parsing method. + # At the end of the parsing run the handler instance, along with it's newly parsed object, + # is returned to the object that originally called for the parsing run (your script/app/whatever). + module Handler + + attr_accessor :stack, :template, :initial_object, :stack_debug + + #SaxParser.install_defaults(self) + + + ### Class Methods ### + + # Main parsing interface (also aliased at SaxParser.parse) + def self.build(io, template=nil, initial_object=nil, parser=nil, options={}) + parser = parser || options[:parser] || BACKEND + parser = get_backend(parser) + (Rfm.log.info "Using backend parser: #{parser}, with template: #{template}") if options[:log_parser] + parser.build(io, template, initial_object) + end + + def self.included(base) + # Add a .build method to the custom handler instance, when the generic Handler module is included. + def base.build(io, template=nil, initial_object=nil) + handler = new(template, initial_object) + handler.run_parser(io) + handler + end + end # self.included + + # Takes backend symbol and returns custom Handler class for specified backend. + def self.get_backend(parser=BACKEND) + (parser = decide_backend) unless parser + if parser.is_a?(String) || parser.is_a?(Symbol) + parser_proc = PARSERS[parser.to_sym][:proc] + parser_proc.call unless parser_proc.nil? || const_defined?((parser.to_s.capitalize + 'Handler').to_sym) + SaxParser.const_get(parser.to_s.capitalize + "Handler") + end + rescue + raise "Could not load the backend parser '#{parser}': #{$!}" + end + + # Finds a loadable backend and returns its symbol. + def self.decide_backend + #BACKENDS.find{|b| !Gem::Specification::find_all_by_name(b[1]).empty? || b[0]==:rexml}[0] + PARSERS.find{|k,v| !Gem::Specification::find_all_by_name(v[:file]).empty? || k == :rexml}[0] + rescue + raise "The xml parser could not find a loadable backend library: #{$!}" + end + + + + ### Instance Methods ### + + def initialize(_template=nil, _initial_object=nil) + @initial_object = case + when _initial_object.nil?; DEFAULT_CLASS.new + when _initial_object.is_a?(Class); _initial_object.new + when _initial_object.is_a?(String) || _initial_object.is_a?(Symbol); SaxParser.get_constant(_initial_object).new + else _initial_object + end + @stack = [] + @stack_debug=[] + @template = get_template(_template) + set_cursor Cursor.new('__TOP__', self).process_new_element + end + + # Takes string, symbol, hash, and returns a (possibly cached) parsing template. + # String can be a file name, yaml, xml. + # Symbol is a name of a template stored in SaxParser@templates (you would set the templates when your app or gem loads). + # Templates stored in the SaxParser@templates var can be strings of code, file specs, or hashes. + def get_template(name) + # dat = templates[name] + # if dat + # rslt = load_template(dat) + # else + # rslt = load_template(name) + # end + # (templates[name] = rslt) #unless dat == rslt + # The above works, but this is cleaner. + TEMPLATES[name] = TEMPLATES[name] && load_template(TEMPLATES[name]) || load_template(name) + end + + # Does the heavy-lifting of template retrieval. + def load_template(dat) + #puts "DAT: #{dat}, class #{dat.class}" + prefix = defined?(TEMPLATE_PREFIX) ? TEMPLATE_PREFIX : '' + #puts "SaxParser::Handler#load_template... 'prefix' is #{prefix}" + rslt = case + when dat.is_a?(Hash); dat + when (dat.is_a?(String) && dat[/^\//]); YAML.load_file dat + when dat.to_s[/\.y.?ml$/i]; (YAML.load_file(File.join(*[prefix, dat].compact))) + # This line might cause an infinite loop. + when dat.to_s[/\.xml$/i]; self.class.build(File.join(*[prefix, dat].compact), nil, {'compact'=>true}) + when dat.to_s[/^<.*>/i]; "Convert from xml to Hash - under construction" + when dat.is_a?(String); YAML.load dat + else DEFAULT_CLASS.new + end + #puts rslt + rslt + end + + def result + stack[0].object if stack[0].is_a? Cursor + end + + def cursor + stack.last + end + + def set_cursor(args) # cursor_object + if args.is_a? Cursor + stack.push(args) + #@stack_debug.push(args.dup.tap(){|c| c.handler = c.handler.object_id; c.parent = c.parent.tag}) + end + cursor + end + + def dump_cursor + stack.pop + end + + def top + stack[0] + end + + def transform(name) + return name unless TAG_TRANSLATION.is_a?(Proc) + TAG_TRANSLATION.call(name.to_s) + end + + # Add a node to an existing element. + def _start_element(tag, attributes=nil, *args) + #puts ["_START_ELEMENT", tag, attributes, args].to_yaml # if tag.to_s.downcase=='fmrestulset' + tag = transform tag + if attributes + # This crazy thing transforms attribute keys to underscore (or whatever). + #attributes = default_class[*attributes.collect{|k,v| [transform(k),v] }.flatten] + # This works but downcases all attribute names - not good. + attributes = DEFAULT_CLASS.new.tap {|hash| attributes.each {|k, v| hash[transform(k)] = v}} + # This doesn't work yet, but at least it wont downcase hash keys. + #attributes = Hash.new.tap {|hash| attributes.each {|k, v| hash[transform(k)] = v}} + end + set_cursor cursor.receive_start_element(tag, attributes) + end + + # Add attribute to existing element. + def _attribute(name, value, *args) + #puts "Receiving attribute '#{name}' with value '#{value}'" + name = transform name + cursor.receive_attribute(name, value) + end + + # Add 'content' attribute to existing element. + def _text(value, *args) + #puts "Receiving text '#{value}'" + #puts RUBY_VERSION_NUM + if RUBY_VERSION_NUM > 1.8 && value.is_a?(String) + #puts "Forcing utf-8" + value.force_encoding('UTF-8') + end + # I think the reason this was here is no longer relevant, so I'm disabeling. + return unless value[/[^\s]/] + cursor.receive_attribute(TEXT_LABEL, value) + end + + # Close out an existing element. + def _end_element(tag, *args) + tag = transform tag + #puts "Receiving end_element '#{tag}'" + cursor.receive_end_element(tag) and dump_cursor + end + + def _doctype(*args) + (args = args[0].gsub(/"/, '').split) if args.size ==1 + _start_element('doctype', :value=>args) + _end_element('doctype') + end + + end # Handler + + + + ##### SAX PARSER BACKEND HANDLERS ##### + + PARSERS[:libxml] = {:file=>'libxml-ruby', :proc => proc do + require 'libxml' + class LibxmlHandler + include LibXML + include XML::SaxParser::Callbacks + include Handler + + def run_parser(io) + parser = case + when (io.is_a?(File) || io.is_a?(StringIO)) + XML::SaxParser.io(io) + when io[/^prefix, :uri=>uri, :xmlns=>namespaces) + # _start_element(name, attributes) + # end + + alias_method :on_start_element, :_start_element + alias_method :on_end_element, :_end_element + alias_method :on_characters, :_text + alias_method :on_internal_subset, :_doctype + end # LibxmlSax + end} + + PARSERS[:nokogiri] = {:file=>'nokogiri', :proc => proc do + require 'nokogiri' + class NokogiriHandler < Nokogiri::XML::SAX::Document + include Handler + + def run_parser(io) + parser = Nokogiri::XML::SAX::Parser.new(self) + parser.parse case + when (io.is_a?(File) || io.is_a?(StringIO)) + io + when io[/^'ox', :proc => proc do + require 'ox' + class OxHandler < ::Ox::Sax + include Handler + + def run_parser(io) + options={:convert_special=>true} + case + when (io.is_a?(File) or io.is_a?(StringIO)); Ox.sax_parse self, io, options + when io.to_s[/^'rexml/document', :proc => proc do + require 'rexml/document' + require 'rexml/streamlistener' + class RexmlHandler + # Both of these are needed to use rexml streaming parser, + # but don't put them here... put them at the _top. + #require 'rexml/streamlistener' + #require 'rexml/document' + include REXML::StreamListener + include Handler + + def run_parser(io) + parser = REXML::Document + case + when (io.is_a?(File) or io.is_a?(StringIO)); parser.parse_stream(io, self) + when io.to_s[/^ + #puts ["\nATTACH_OBJECT._attach_object", "self.class: #{self.class}", "obj.class: #{obj.class}", "obj.to_s: #{obj.to_s}", "args: #{args}"] + options = ATTACH_OBJECT_DEFAULT_OPTIONS.merge(args.last.is_a?(Hash) ? args.pop : {}){|key, old, new| new || old} + # name = (args[0] || options[:name]) + # delimiter = (args[1] || options[:delimiter]) + prefs = (args[2] || options[:prefs]) + # type = (args[3] || options[:type]) + return if (prefs=='none' || prefs=='cursor') #['none', 'cursor'].include? prefs ... not sure which is faster. + self._merge_object!( + obj, + args[0] || options[:name] || 'unknown_name', + args[1] || options[:delimiter], + prefs, + args[3] || options[:type], + options + ) + + # case + # when prefs=='none' || prefs=='cursor'; nil + # when name + # self._merge_object!(obj, name, delimiter, prefs, type, options) + # else + # self._merge_object!(obj, 'unknown_name', delimiter, prefs, type, options) + # end + #puts ["\nATTACH_OBJECT RESULT", self.to_yaml] + #puts ["\nATTACH_OBJECT RESULT PORTALS", (self.portals.to_yaml rescue 'no portals')] + end + + # Master method to merge any object with this object + def _merge_object!(obj, name, delimiter, prefs, type, options={}) + #puts ["\n-----OBJECT._merge_object", self.class, (obj.to_s rescue obj.class), name, delimiter, prefs, type.capitalize, options].join(', ') + if prefs=='private' + _merge_instance!(obj, name, delimiter, prefs, type, options) + else + _merge_shared!(obj, name, delimiter, prefs, type, options) + end + end + + # Merge a named object with the shared instance variable of self. + def _merge_shared!(obj, name, delimiter, prefs, type, options={}) + shared_var = instance_variable_get("@#{options[:shared_variable_name]}") || instance_variable_set("@#{options[:shared_variable_name]}", options[:default_class].new) + #puts "\n-----OBJECT._merge_shared: self '#{self.class}' obj '#{obj.class}' name '#{name}' delimiter '#{delimiter}' type '#{type}' shared_var '#{options[:shared_variable_name]} - #{shared_var.class}'" + # TODO: Figure this part out: + # The resetting of shared_variable_name to 'attributes' was to fix Asset.field_controls (it was not able to find the valuelive name). + # I think there might be a level of hierarchy that is without a proper cursor model, when using shared variables & object delimiters. + shared_var._merge_object!(obj, name, delimiter, nil, type, options.merge(:shared_variable_name=>ATTACH_OBJECT_DEFAULT_OPTIONS[:shared_variable_name])) + end + + # Merge a named object with the specified instance variable of self. + def _merge_instance!(obj, name, delimiter, prefs, type, options={}) + #puts ["\nOBJECT._merge_instance!", "self.class: #{self.class}", "obj.class: #{obj.class}", "name: #{name}", "delimiter: #{delimiter}", "prefs: #{prefs}", "type: #{type}", "options.keys: #{options.keys}", '_end_merge_instance!'] #.join(', ') + rslt = if instance_variable_get("@#{name}") || delimiter + if delimiter + delimit_name = obj._get_attribute(delimiter, options[:shared_variable_name]).to_s.downcase + #puts ["\n_setting_with_delimiter", delimit_name] + #instance_variable_set("@#{name}", instance_variable_get("@#{name}") || options[:default_class].new)[delimit_name]=obj + # This line is more efficient than the above line. + instance_variable_set("@#{name}", options[:default_class].new) unless instance_variable_get("@#{name}") + + # This line was active in 3.0.9.pre01, but it was inserting each portal array as an element in the array, + # after all the Rfm::Record instances had been added. This was causing an potential infinite recursion situation. + # I don't think this line was supposed to be here - was probably an older piece of code. + #instance_variable_get("@#{name}")[delimit_name]=obj + + #instance_variable_get("@#{name}")._merge_object!(obj, delimit_name, nil, nil, nil) + # Trying to handle multiple portals with same table-occurance on same layout. + # In parsing terms, this is trying to handle multiple elements who's delimiter field contains the SAME delimiter data. + instance_variable_get("@#{name}")._merge_delimited_object!(obj, delimit_name) + else + #puts ["\_setting_existing_instance_var", name] + if name == options[:text_label] + instance_variable_get("@#{name}") << obj.to_s + else + instance_variable_set("@#{name}", [instance_variable_get("@#{name}")].flatten << obj) + end + end + else + #puts ["\n_setting_new_instance_var", name] + instance_variable_set("@#{name}", obj) + end + + # NEW + _create_accessor(name) if (options[:create_accessors] & ['all','private']).any? + + rslt + end + + def _merge_delimited_object!(obj, delimit_name) + #puts "MERGING DELIMITED OBJECT self #{self.class} obj #{obj.class} delimit_name #{delimit_name}" + + case + when self[delimit_name].nil?; self[delimit_name] = obj + when self[delimit_name].is_a?(Hash); self[delimit_name].merge!(obj) + when self[delimit_name].is_a?(Array); self[delimit_name] << obj + else self[delimit_name] = [self[delimit_name], obj] + end + end + + # Get an instance variable, a member of a shared instance variable, or a hash value of self. + def _get_attribute(name, shared_var_name=nil, options={}) + return unless name + #puts ["\n\n", self.to_yaml] + #puts ["OBJECT_get_attribute", self.class, self.instance_variables, name, shared_var_name, options].join(', ') + (shared_var_name = options[:shared_variable_name]) unless shared_var_name + + rslt = case + when self.is_a?(Hash) && self[name]; self[name] + when ((var= instance_variable_get("@#{shared_var_name}")) && var[name]); var[name] + else instance_variable_get("@#{name}") + end + + #puts "OBJECT#_get_attribute: name '#{name}' shared_var_name '#{shared_var_name}' options '#{options}' rslt '#{rslt}'" + rslt + end + + # # We don't know which attributes are shared, so this isn't really accurate per the options. + # # But this could be useful for mass-attachment of a set of attributes (to increase performance in some situations). + # def _create_accessors options=[] + # options=[options].flatten.compact + # #puts ['CREATE_ACCESSORS', self.class, options, ""] + # return false if (options & ['none']).any? + # if (options & ['all', 'private']).any? + # meta = (class << self; self; end) + # meta.send(:attr_reader, *instance_variables.collect{|v| v.to_s[1,99].to_sym}) + # end + # if (options & ['all', 'shared']).any? + # instance_variables.collect{|v| instance_variable_get(v)._create_accessors('hash')} + # end + # return true + # end + + # NEW + def _create_accessor(name) + #puts "OBJECT._create_accessor '#{name}' for Object '#{self.class}'" + meta = (class << self; self; end) + meta.send(:attr_reader, name.to_sym) + end + + # Attach hash as individual instance variables to self. + # This is for manually attaching a hash of attributes to the current object. + # Pass in translation procs to alter the keys or values. + def _attach_as_instance_variables(hash, options={}) + #hash.each{|k,v| instance_variable_set("@#{k}", v)} if hash.is_a? Hash + key_translator = options[:key_translator] + value_translator = options[:value_translator] + #puts ["ATTACH_AS_INSTANCE_VARIABLES", key_translator, value_translator].join(', ') + if hash.is_a? Hash + hash.each do |k,v| + (k = key_translator.call(k)) if key_translator + (v = value_translator.call(k, v)) if value_translator + instance_variable_set("@#{k}", v) + end + end + end + +end # Object + +class Array + def _merge_object!(obj, name, delimiter, prefs, type, options={}) + #puts ["\n+++++ARRAY._merge_object", self.class, (obj.to_s rescue obj.class), name, delimiter, prefs, type, options].join(', ') + case + when prefs=='shared' || type == 'attribute' && prefs.to_s != 'private' ; _merge_shared!(obj, name, delimiter, prefs, type, options) + when prefs=='private'; _merge_instance!(obj, name, delimiter, prefs, type, options) + else self << obj + end + end +end # Array + +class Hash + + def _merge_object!(obj, name, delimiter, prefs, type, options={}) + #puts ["\n*****HASH._merge_object", "type: #{type}", "name: #{name}", "self.class: #{self.class}", "new_obj: #{(obj.to_s rescue obj.class)}", "delimiter: #{delimiter}", "prefs: #{prefs}", "options: #{options}"] + output = case + when prefs=='shared' + _merge_shared!(obj, name, delimiter, prefs, type, options) + when prefs=='private' + _merge_instance!(obj, name, delimiter, prefs, type, options) + when (self[name] || delimiter) + rslt = if delimiter + delimit_name = obj._get_attribute(delimiter, options[:shared_variable_name]).to_s.downcase + #puts "MERGING delimited object with hash: self '#{self.class}' obj '#{obj.class}' name '#{name}' delim '#{delimiter}' delim_name '#{delimit_name}' options '#{options}'" + self[name] ||= options[:default_class].new + + #self[name][delimit_name]=obj + # This is supposed to handle multiple elements who's delimiter value is the SAME. + self[name]._merge_delimited_object!(obj, delimit_name) + else + if name == options[:text_label] + self[name] << obj.to_s + else + self[name] = [self[name]].flatten + self[name] << obj + end + end + _create_accessor(name) if (options[:create_accessors].to_a & ['all','shared','hash']).any? + + rslt + else + rslt = self[name] = obj + _create_accessor(name) if (options[:create_accessors] & ['all','shared','hash']).any? + rslt + end + #puts ["\n*****HASH._merge_object! RESULT", self.to_yaml] + #puts ["\n*****HASH._merge_object! RESULT PORTALS", (self.portals.to_yaml rescue 'no portals')] + output + end + + # def _create_accessors options=[] + # #puts ['CREATE_ACCESSORS_for_HASH', self.class, options] + # options=[options].flatten.compact + # super and + # if (options & ['all', 'hash']).any? + # meta = (class << self; self; end) + # keys.each do |k| + # meta.send(:define_method, k) do + # self[k] + # end + # end + # end + # end + + def _create_accessor(name) + #puts "HASH._create_accessor '#{name}' for Hash '#{self.class}'" + meta = (class << self; self; end) + meta.send(:define_method, name) do + self[name] + end + end + +end # Hash + + + + +# #### NOTES and TODOs #### +# +# done: Move test data & user models to spec folder and local_testing. +# done: Create special file in local_testing for experimentation & testing - will have user models, grammar-yml, calling methods. +# done: Add option to 'compact' unnecessary or empty elements/attributes - maybe - should this should be handled at Model level? +# na : Separate all attribute options in yml into 'attributes:' hash, similar to 'elements:' hash. +# done: Handle multiple 'text' callbacks for a single element. +# done: Add options for text handling (what to name, where to put). +# done: Fill in other configuration options in yml +# done: Clean_elements doesn't work if elements are non-hash/array objects. Make clean_elements work with object attributes. +# TODO: Clean_elements may not work for non-hash/array objecs with multiple instance-variables. +# na : Clean_elements may no longer work with a globally defined 'compact'. +# TODO: Do we really need Cursor#top and Cursor#stack ? Can't we get both from handler.stack. Should we store handler in cursor inst var @handler? +# TODO: When using a non-hash/array object as the initial object, things get kinda srambled. +# See Rfm::Connection, when sending self (the connection object) as the initial_object. +# done: 'compact' breaks badly with fmpxmllayout data. +# na : Do the same thing with 'ignore' as you did with 'attach', so you can enable using the 'attributes' array of the yml model. +# done: Double-check that you're pointing to the correct model/submodel, since you changed all helper-methods to look at curent-model by default. +# done: Make sure all method calls are passing the model given by the calling-method. +# abrt: Give most of the config options a global possibility. (no true global config, but top-level acts as global for all immediate submodels). +# na : Block attachment methods from seeing parent if parent isn't the current objects true parent (how?). +# done: Handle attach: hash better (it's not specifically handled, but declaring it will block a parents influence). +# done: CaseInsensitiveHash/IndifferentAccess is not working for sax parser. +# na : Give the yml (and xml) doc the ability to have a top-level hash like "fmresultset" or "fmresultset_yml" or "fmresultset_xml", +# then you have a label to refer to it if you load several config docs at once (like into a Rfm::SaxParser::TEMPLATES constant). +# Use an array of accepted model-keys to filter whether loaded template is a named-model or actual model data. +# done: Load up all template docs when Rfm loads, or when Rfm::SaxParser loads. For RFM only, not for parser module. +# done: Move SaxParser::Handler class methods to SaxParser, so you can do Rfm::SaxParser.parse(io, backend, template, initial_object) +# done: Switch args order in .build methods to (io, template, initial_object, backend) +# done: Change "grammar" to "template" in all code +# done: Change 'cursor._' methods to something more readable, since they will be used in Rfm and possibly user models. +# done: Split off template loading into load_templates and/or get_templates methods. +# TODO: Something is downcasing somewhere - see the fmpxmllayout response. Looks like 'compact' might have something to do with it. +# done: Make attribute attachment default to individual. +# done: 'attach: shared' doesnt work yet for elements. +# na : Arrays should always get elements attached to their records and attributes attached to their instance variables. +# done: Merge 'ignore: self, elements, attributes' config into 'attach: ignore, attach_elements: ignore, attach_attributes: ignore'. +# done: Consider having one single group of methods to attach all objects (elements OR attributes OR text) to any given parent object. +# na : May need to store 'ignored' models in new cursor, with the parent object instead of the new object. Probably not a good idea +# done: Fix label_or_tag for object-attachment. +# done: Fix delineate_with_hash in parsing of resultset field_meta (should be hash of hashes, not array of hashes). +# done: Test new parser with raw data from multiple sources, make sure it works as raw. +# done: Make sure single-attribute (or text) handler has correct objects & models to work with. +# na : Rewrite attach_to_what? logic to start with base_object type, then have sub-case statements for the rest. +# done: Build backend-gem loading scheme. Eliminate gem 'require'. Use catch/throw like in XmlParser. +# done: Splash_sax.rb is throwing error when loading User.all when SaxParser.backend is anything other than :ox. +# This is probably related to the issue of incomming attributes (that are after the incoming start_element) not knowing their model. +# Some multiple-text attributes were tripping up delineate_with_hash, so I added some fault-tollerance to that method. +# But those multi-text attributes are still way ugly when passed by libxml or nokogiri. Ox & Rexml are fine and pass them as one chunk. +# Consider buffering all attributes & text until start of new element. +# YAY : I bufffered all attributes & text, and the problem is solved. +# na : Can't configure an attribute (in template) if it is used in delineate_with_hash. (actually it works if you specifiy the as_name: correctly). +# TODO: Some configurations in template cause errors - usually having to do with nil. See below about eliminating all 'rescue's . +# done: Can't supply custom class if it's a String (but an unspecified subclass of plain Object works fine!?). +# TODO: Attaching non-hash/array object to Array will thro error. Is this actually fixed? +# done?: Sending elements with subelements to Shared results in no data attached to the shared var. +# TODO: compact is not working for fmpxmllayout-error. Consider rewrite of 'compact' method, or allow compact to work on end_element with no matching tag. +# mabe: Add ability to put a regex in the as_name parameter, that will operate on the tag/label/name. +# TODO: Optimize: +# Use variables, not methods. +# Use string interpolation not concatenation. +# Use destructive! operations (carefully). Really? +# Get this book: http://my.safaribooksonline.com/9780321540034?portal=oreilly +# See this page: http://www.igvita.com/2008/07/08/6-optimization-tips-for-ruby-mri/ +# done: Consider making default attribute-attachment shared, instead of instance. +# done: Find a way to get SaxParser defaults into core class patches. +# done: Check resultset portal_meta in Splash Asset model for field-definitions coming out correct according to sax template. +# done: Make sure Rfm::Metadata::Field is being used in portal-definitions. +# done: Scan thru Rfm classes and use @instance variables instead of their accessors, so sax-parser does less work. +# done: Change 'instance' option to 'private'. Change 'shared' to . +# done: Since unattached elements won't callback, add their callback to an array of procs on the current cursor at the beginning +# of the non-attached tag. +# TODO: Handle check-for-errors in non-resultset loads. +# abrt: Figure out way to get doctype from nokogiri. Tried, may be practically impossible. +# TODO: Clean up sax code so that no 'rescue' is needed - if an error happens it should be a legit error outside of SaxParser. +# TODO: Allow attach:none when using handler. + + diff --git a/lib/rfm/utilities/scope.rb b/lib/rfm/utilities/scope.rb new file mode 100644 index 00000000..ed0c0bd1 --- /dev/null +++ b/lib/rfm/utilities/scope.rb @@ -0,0 +1,95 @@ +require 'rfm' +require 'rfm/base' + +module Rfm + module Scope + + SCOPE = Proc.new {[]} + + # Add scoping to Rfm::Base class methods for querying fmp records. + # Usage: + # class MyModel < Rfm::Base + # SCOPE = proc { |optional-scope-args| } + # end + # + # Optionally pass :scope=>fmp-request-hash-or-array or :scope_args=>anything + # in the FMP request options hash, to be used at scoping time, + # instead of above scope constant. + + def find(*args) + new_args = apply_scope(args) + super(*new_args) + end + + def count(*args) + new_args = apply_scope(args) + super(*new_args) + end + + def delineate_query(*request) + options = (request.last.is_a?(Hash) && request.size > 1) ? request.pop : {} + query = request.pop || {} + action = request.pop || (query.size==0 ? :all : :find) + #puts "DELINEATE_QUERY action:#{action} query:#{query} options:#{options}" + [action, query, options] + end + + # Mix scope requests with user requests with a constraining AND logic. + # Also handles request options. + def apply_scope(target_request, raw_scope=nil) + # Separate target_request into query, opts, with query always enclosed in array. + target_query, target_opts = delineate_query(*target_request)[1..2].inject{|q,o| [[q].flatten, o]} + # Retrieve :scope_args, if any, from target_request. + scope_args = target_opts.delete(:scope_args) || self + # Get raw scope from several possible sources + raw_scope = raw_scope || target_opts.delete(:scope) || self::SCOPE + # Compile raw scope from Proc, if necessary, into full scope_request + scope_request = [raw_scope.is_a?(Proc) ? raw_scope.call(scope_args) : raw_scope].flatten(1).compact + # Separate scope_request into query, opts, with query always enclosed in array. + scope_query, scope_opts = delineate_query(*scope_request)[1..2].inject{|q,o| [[q].flatten, o]} + # Extract scope & target omits into discrete arrays. + scope_omits, target_omits = [],[] + target_query.delete_if{|q| target_omits.push(q) if q[:omit]} + scope_query.delete_if{|q| scope_omits.push(q) if q[:omit]} + + #puts "APPLY_SCOPE TARGET query:#{target_query} omits:#{target_omits} opts:#{target_opts}" + #puts "APPLY_SCOPE SCOPE query:#{scope_query} omits:#{scope_omits} opts:#{scope_opts}" + + # Return original request if no scoping can be done. + return target_request unless (target_query.is_a?(Array) || target_query.is_a?(Hash)) && (scope_query.size > 0 || scope_omits.size > 0 || scope_opts.size > 0) + + # Create product of target & scope + scoped_queries = case + when (target_query.any? && !scope_query.any?); target_query + when (scope_query.any? && !target_query.any?); scope_query + #else target_query.product(scope_query).map{|a,b| (b[:omit] || a[:omit]) ? nil : a.merge(b)}.compact + else target_query.product(scope_query).map{|a,b| a.merge(b)} + end + scoped_omits = (target_omits | scope_omits) + + #puts "APPLY_SCOPE OUTPUT #{[scoped_queries | scoped_omits, target_opts.merge(scope_opts)]}" + + [scoped_queries | scoped_omits, target_opts.merge(scope_opts)] + end + + # def self.extended(base) + # puts "Extending #{base} with Scope" + # end + + end # Scope + + + class Base + SCOPE = Scope::SCOPE + + class << self + # When Rfm::Base is inherited, the inheritor will extend this Scope module + alias_method :inherited_orig, :inherited + def inherited(model) + super(model) + model.send :extend, Scope + end + end + end + +end # Rfm diff --git a/lib/rfm/version.rb b/lib/rfm/version.rb new file mode 100644 index 00000000..220ba8b3 --- /dev/null +++ b/lib/rfm/version.rb @@ -0,0 +1,27 @@ +module Rfm + VERSION_DEFAULT = 'none' + VERSION = File.read(PATH + '/rfm/VERSION').lines.first.chomp rescue VERSION_DEFAULT + + VERSION.instance_eval do + def components + VERSION.split('.') + end + + def major + components[0] + end + + def minor + components[1] + end + + def patch + components[2] + end + + def build + components[3] + end + end + +end diff --git a/spec/active_model_lint.rb b/spec/active_model_lint.rb new file mode 100644 index 00000000..7ca2b5a2 --- /dev/null +++ b/spec/active_model_lint.rb @@ -0,0 +1,38 @@ + +# Copied from spec run deprecation warning on ruby 2.1.3@rfm: +# +# The semantics of `described_class` in a nested `describe ` +# example group are changing in RSpec 3. In RSpec 2.x, `described_class` +# would return the outermost described class (Rfm::Base). +# In RSpec 3, it will return the innermost described class (TestModel). +# In general, we recommend not describing multiple classes or objects in a +# nested manner as it creates confusion. +# +# To make your code compatible with RSpec 3, change from `described_class` to a reference +# to `TestModel`, or change the arg of the inner `describe` to a string. +# (Called from /Users/wbr/Documents/programming/gitprojects/GinjoRfm.git/spec/active_model_lint.rb:20:in `model') + + + +shared_examples_for "ActiveModel" do + require 'minitest' + include Minitest::Assertions + include ActiveModel::Lint::Tests + + attr_accessor :assertions + + def initialize(*args) + self.assertions = 0 + super(*args) + end + + ActiveModel::Lint::Tests.public_instance_methods.map{|m| m.to_s}.grep(/^test/).each do |m| + example m.gsub('_',' ') do + send m + end + end + + def model + subject + end +end diff --git a/spec/data/data_fmpxmlresult.xml b/spec/data/data_fmpxmlresult.xml new file mode 100644 index 00000000..30e071d8 --- /dev/null +++ b/spec/data/data_fmpxmlresult.xml @@ -0,0 +1 @@ + 0 TJKirk jazz 10/6/1996 10/6/1996 To Get Alphebet Soup jazz 10/6/1996 10/6/1996 To Get Mingus Amungus jazz 10/6/1996 10/6/1996 To Get Papa's Culture jazz 10/6/1996 10/6/1996 To Get Charlie Hunter jazz 10/6/1996 10/6/1996 To Get Ornette Coleman jazz 10/6/1996 10/6/1996 To Get Broun Fellinis jazz 10/6/1996 10/6/1996 To Get DogSlyde jazz 10/6/1996 10/6/1996 To Get John Tchicai & the Archetypes jazz 10/6/1996 10/6/1996 To Get Jimmy Smith jazz 10/6/1996 10/6/1996 To Get Grassy Knoll jazz 10/6/1996 10/6/1996 To Get Robert Stewart jazz 10/6/1996 10/6/1996 To Get Josh Jones Quartet jazz 10/6/1996 10/6/1996 To Get Dave Ellis jazz formerly of Charlie Hunter Trio 10/6/1996 10/6/1996 To Get Ann Dyer & No Good Time Fairies jazz 10/6/1996 10/6/1996 To Get Oranj Symphonette jazz Henri Mancini revisionists 10/6/1996 10/6/1996 To Get Will Bernard's Quartet jazz 10/6/1996 10/6/1996 To Get DJ Cheb I Saabah jazz 10/6/1996 10/6/1996 To Get DJ Andrew jazz 10/6/1996 10/6/1996 To Get Bob Green - (Grassy Knoll?) jazz 10/6/1996 10/6/1996 To Get De La Soul rap 10/6/1996 10/6/1996 To Get Headless Chickens techno NewZealand/Australia 10/6/1996 3/4/1997 Current MouthMusic techno Scottland 10/6/1996 10/6/1996 To Get Orb techno 10/6/1996 10/6/1996 To Get Strawpeople techno NewZealand/Australia 10/6/1996 3/4/1997 Current Broadcast Sarah Mclachlan rock 10/7/1996 3/4/1997 To Get Gangstar hiphop 3/4/1997 3/4/1997 To Get Luscious Jackson pop Produced by Daniel Lenois 3/24/1997 3/24/1997 To Get the new one Marlui Miranda trip Brazillian tripy 3/24/1997 3/24/1997 To Get ? Ultrasound trip 3/24/1997 3/24/1997 To Get Tactile vs. the Null Set PJ Harvey rock 3/24/1997 3/24/1997 To Get Rote Bridge Crossing Jane Ira Bloom jazz Sax Jazz 3/24/1997 3/24/1997 To Get Ten Second Dynasty trip Floydian trippy rock 3/24/1997 3/24/1997 To Get Azellia Snail trip moody dreamy guitar with alto femail corus 3/24/1997 3/24/1997 To Get Psychedelicious trip compilation 3/24/1997 3/24/1997 To Get Guitar Voodo - song Vapor Trail 3/24/1997 3/24/1997 To Get 3/24/1997 3/24/1997 To Get Starving for Starlight - song Quagmire rock heavy etherial rock 3/24/1997 3/24/1997 To Get Grady Sisters rock 3/24/1997 3/24/1997 To Get voice of the world Morphine sax rock 3/24/1997 3/24/1997 To Get Symetries trip like tangerine dream 3/24/1997 3/24/1997 Emit To Get Emit2295 Tangerine Dream trip 3/24/1997 3/24/1997 To Get Rubicon Porcupine Tree trip new age Floyd 3/24/1997 3/24/1997 To Get Django Reinhardt 3/24/1997 3/24/1997 To Get Pat Metheny jazz Dec '96 w/Lyle Mays 3/24/1997 1/23/2000 Finally got it. Quartet? John Fehy and Culdesac experimental 11/5/1997 11/5/1997 To Get Epiphany of... Vandergraph Generator trip heard on 90.7 Sounds like old trippy Bowie 5/14/1998 5/14/1998 ? To Get ? Alek Nakhita african/french/40s 1/6/1999 1/6/1999 To Get ? Sountrack from Manella film. african/french/40s 1/6/1999 1/6/1999 To Get ? Guy Davis Blues Good Slide and Harmonica Blues. Has an albumn where he tells a story and plays. 5/18/1999 5/18/1999 To Get Concentric Opera techno world Bjork meets Orb 1/23/2000 2/21/2000 To Get song: black is the color Stereolab 2/21/2000 2/21/2000 To Get Ron Carter Sextet jazz Jazzy mellow funky fusion 2/21/2000 2/21/2000 To Get the new one with Steve___ on piano. UK rock 2/25/2000 2/25/2000 To Get Steve Hacket rock 2/25/2000 2/25/2000 To Get Il Gordiero Harmonico classical plays Bach- some concerto #2 4/26/2001 4/26/2001 To Get bruce cannon space Bay area pedal steel 4/26/2001 4/26/2001 To Get Bill Frissel new 2/2001 4/26/2001 4/26/2001 To Get Bobo Stenson - pianist piano new CD 4/26/2001 4/26/2001 To Get Skatelites ska good ska band 4/26/2001 4/26/2001 To Get Elvin Jones 4/26/2001 4/26/2001 To Get Main Force Cyrus... Piano 4/26/2001 4/26/2001 To Get Johnny Hodges / Duke Elington 4/26/2001 4/26/2001 To Get Innerzone Orchestra jazz from JazzyBeats - Flaresound mp3 radio 4/27/2001 4/27/2001 To Get Basic Math - song Funk D'void 1/24/2002 1/24/2002 To Get Diabla Eva Cassidy folk One of Pat's favorites 1/25/2002 1/25/2002 To Get Songbird Phenomena trip 5/15/2002 5/15/2002 To Get Bill Hyde jazz 5/15/2002 5/15/2002 To Get Quixotic 5/15/2002 5/15/2002 Husker Du 5/15/2002 5/15/2002 Flaming Lips 5/15/2002 5/15/2002 asg@SDRproductions.com trip 5/15/2002 5/15/2002 'Blue Room' show Beatless trip trippy beat stuff 5/15/2002 5/15/2002 To Get Matrix electronica fast groovy electronica 5/15/2002 5/15/2002 To Get Irresistable Force ambient/techno ambient with great percussion 5/15/2002 5/15/2002 To Get Power T Spigot trip 5/15/2002 5/15/2002 To Get Molly Bolts pop good pop band 5/15/2002 5/15/2002 To Get Afro-Mystic ambient/techno 5/15/2002 5/15/2002 To Get Infinite Rhythm fila brazillia ambient/techno slow warm trippy. Not Brazilian. 5/15/2002 5/15/2002 To Get soft music under stars - on the Mess album happy campers electronica 5/15/2002 5/15/2002 To Get kolida ambient/techno nice female vocals 5/15/2002 5/15/2002 To Get angels fly Thunderball dance/trance 5/15/2002 5/15/2002 To Get Domino London Funk Allstars funk 5/15/2002 5/15/2002 To Get The Chase \ No newline at end of file diff --git a/spec/data/data_with_portals_fmpxmlresult.xml b/spec/data/data_with_portals_fmpxmlresult.xml new file mode 100644 index 00000000..c1edbb44 --- /dev/null +++ b/spec/data/data_with_portals_fmpxmlresult.xml @@ -0,0 +1 @@ + 0 300 16150 5 email Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 4 jabel@axiantgroup.com Abel, John & Sarah 88 300 300 200 100 300 400 300 300 3322 3590 3632 3762 4054 4318 dues assessment dues buoy field assessment Dues Dues 7/21/2008 5/7/2009 6/25/2009 11/24/2009 9/30/2010 7/13/2011 161 Estates Drive Piedmont Abel, John & Sarah CA 94611 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 4 4 4 4 4 4 4 Dues Assessment Dues Assessment Dues Dues Dues 4 1 88 300 16150 5 email Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 29 cgoetz1480@hotmail.com Allen, Dorothy 55 300 300 200 300 100 400 300 300 8966 9060 9184 3604 3774 9484 dues dues assessment buoy field assessment dues dues 8/21/2008 6/8/2009 7/23/2009 12/2/2009 7/22/2010 9/12/2011 1480 Pear Drive Concord Goetz, Carolyn & Rick CA 94518 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 29 29 29 29 29 29 29 Dues Assessment Dues Assessment Dues Dues Dues 29 1 55 300 16150 5 usps Assessment reminder & past-due statement emailed 2/10/2010 - email address failed. Assessment reminder & past-due statement printed 2/11/2010. Assessment reminder & past-due statement mailed 2/11/2010. Dues letter sent 8/2/2010 Statement printed 4/4/2011 Statement sent 4/5/2011 Statement sent 5/1/2012 9 mbsschart@aol.com Bosschart, Marc & Michele 68 300 300 300 300 400 300 300 1074 1095 1111 ? 1161 dues & assessment dues buoy field assessment Dues dues 5/7/2009 10/14/2009 3/10/2010 3/10/2011 1/27/2012 147 Elm Ave Hillsborough Bosschart, Marc & Michele CA 94010 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 9 9 9 9 9 9 9 Dues Assessment Dues Assessment Dues Dues Dues 9 1 68 500 16150 5 email Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 31 cattersond@aol.com Catterson, Don 35 300 300 200 100 300 400 400 2292 4652 4610 2428 2495 dues assessment dues buoy field assessment buoy field assessment overpay, apply to 2010 dues 7/21/2008 12/29/2008 7/23/2009 11/24/2009 5/27/2010 386 Red River Rd. Palm Desert Catterson, Don CA 92211 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 31 31 31 31 31 31 31 Dues Assessment Dues Assessment Dues Dues Dues 31 1 35 300 16150 5 email Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 27 tconner@thoits.com Conner, Margaret 75 300 300 200 100 300 700 300 5042 2704 2844 0045638120 0058386469 dues assessment dues & buoy field assessment dues & buoy field assessment dues 8/21/2008 5/7/2009 11/6/2009 12/2/2009 5/24/2011 611 Devon Dr. Hillsboro Conner, Margaret CA 94010 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 27 27 27 27 27 27 27 Dues Assessment Dues Assessment Dues Dues Dues 27 1 75 900 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement resent 4/7/2011 Statement sent 5/1/2012 16 sleuthantiques@comcast.net Damner, Bert & Sisi 36 300 300 300 300 400 11250 11536 11808 dues & assessment dues buoy field assessment 8/21/2008 6/8/2009 3/10/2010 1 San Carlos Ave Sausalito Damner, Bert & Sisi CA 94965 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 16 16 16 16 16 16 16 Dues Assessment Dues Assessment Dues Dues Dues 16 1 36 1300 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 30 degerland@aol.com Egerland, Jens & Debbie 45 300 300 300 300 4742 4336 dues & assessment dues 5/7/2009 2/18/2009 2671 Huntington Road Sacramento Egerland, Jens & Debbie CA 95864 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 30 30 30 30 30 30 30 Dues Assessment Dues Assessment Dues Dues Dues 30 1 45 700 16150 5 usps Assessment reminder & past-due statement printed 2/8/2010 Assessment reminder & past-due statement mailed 2/10/2010 Dues letter sent 8/2/2010 Statement printed 4/4/2011 Statement sent 4/5/2011 Statement sent 5/1/2012 17 Figone, Lewis 32 300 300 200 100 500 300 300 6526 6625 6784 0142 0299 dues assessment dues Dues Dues 7/21/2009 12/29/2008 8/4/2009 9/30/2010 5/24/2011 POBox 277 El Cerrito Figone, Lewis CA 94530 200 200 100 300 400 300 300 300 2007-08 dues late Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 17 17 17 17 17 17 17 17 begin Dues Assessment Dues Assessment Dues Dues Dues 17 1 32 600 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 20 renegade@gmail.com Folk, Roy & Daisy 20 300 300 200 400 400 300 6790 6909 6865 7381 dues dues buoy field assessment Dues 7/21/2008 7/11/2009 5/5/2010 6/14/2010 15314 Sobey Rd. Saratoga Folk, Roy & Daisy CA 95070 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 20 20 20 20 20 20 20 Dues Assessment Dues Assessment Dues Dues Dues 20 1 20 500 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 24 dgarello@msn.com Garello, Dave & Nikki 15 300 300 200 300 300 400 400 3313 3418 3527 3787 2724 dues 2007-8 dues + assessment dues buoy field assessment dues 7/21/2008 12/29/2008 6/8/2009 3/19/2010 7/13/2011 POBox 1042 Pebble Beach Garello, Dave & Nikki CA 93953 200 100 200 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders 2007-08 dues late (lost check?) Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 7/1/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 24 24 24 24 24 24 24 24 Dues Assessment begin Dues Assessment Dues Dues Dues 24 1 15 300 16150 5 usps Dues letter sent 8/2/2010 Statement printed 4/4/2011 Statement sent 4/5/2011 Statement sent 5/1/2012 22 Gilbert, Lillian 12 300 300 1000 300 300 2593 3052 3223 dues & buoy field assessment Dues Dues 11/24/2009 9/30/2010 9/12/2011 2685 Lakeside Dr Reno Gilbert, Lillian NV 89509 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 22 22 22 22 22 22 22 Dues Assessment Dues Assessment Dues Dues Dues 22 1 12 300 16150 5 usps Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 25 Grant.Matthews@morganstanleysmithbarney.com Grant Matthews 95 300 300 300 300 700 300 860 0000881475 1287 547125207 dues & assessment dues dues & assessment dues (First American Title) 7/19/2008 10/14/2009 6/9/2011 7/13/2011 1720 Poppy Avenue Menlo Park Grant Matthews CA 94025 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 25 25 25 25 25 25 25 Dues Assessment Dues Assessment Dues Dues Dues 25 1 95 300 16150 5 usps Dues letter sent 8/2/2010 Statement printed 4/4/2011 Statement sent 4/5/2011 Statement sent 5/1/2012 15 sherylgranzella@yahoo.com Granzella, Richard & Jearene 40 300 300 200 300 -300 800 300 300 3932 94083090 94083090 4082 110912791 115193038 dues dues returned check dues & buoy field assessment Dues Dues 7/21/2008 8/4/2009 10/14/2009 11/10/2009 12/16/2010 5/24/2011 5419 Heavenly Ridge Lane Richmond Sheryl Granzella CA 94803 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 15 15 15 15 15 15 15 Dues Assessment Dues Assessment Dues Dues Dues 15 1 40 300 16150 5 email Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 32 wgriffith@tcv.com Griffith, Calla & Will 25 300 300 100 500 200 400 300 300 1077 0097910655 0005706716 1246 111277874 132284131 assessment dues dues buoy field assessment Dues Dues 12/29/2008 6/29/2009 7/11/2009 11/10/2009 12/16/2010 5/24/2011 147 Laurel Street Atherton Griffith, Calla & Will CA 94027 200 200 100 300 400 300 300 300 2007-08 dues late Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 32 32 32 32 32 32 32 32 begin Dues Assessment Dues Assessment Dues Dues Dues 32 1 25 1400 16150 5 usps Assessment reminder & past-due statement printed 2/8/2010 Assessment reminder & past-due statement mailed 2/10/2010 Dues letter sent 8/2/2010 Statement printed 4/4/2011 Statement sent 4/5/2011 Statement sent 5/1/2012 7 Grimes, James & Nancy 76 300 300 200 200 100 4062 1018 1019 dues dues assessment 7/21/2008 7/23/2009 7/23/2009 8036 Acapella Circle Antelope Grimes, James & Nancy CA 95843 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 7 7 7 7 7 7 7 Dues Assessment Dues Assessment Dues Dues Dues 7 1 76 300 16150 5 email Need address 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 18 heff@heffgroup.com, annh@heffins.com Heffernan 28 300 300 200 100 100 200 1000 2348 2397 2430 2451 7446 dues assessment dues dues Dues & Assessment 2/18/2009 5/7/2009 6/8/2009 7/23/2009 5/24/2011 1350 Carlback Ave. #200 Walnut Creek Heffernan CA 94596 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 18 18 18 18 18 18 18 Dues Assessment Dues Assessment Dues Dues Dues 18 1 28 300 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Single statement sent by request 3/2011 Statement sent 4/6/2011 Statement sent 5/1/2012 2 cdksilacci@aol.com, lshicks1@aol.com Hicks, Clair & Linda 96 300 300 200 100 300 700 300 772 4846 5031 5759 5807 dues assessment dues Dues & Assessment Dues 7/21/2008 5/7/2009 7/23/2009 5/24/2011 7/13/2011 202 Alameda Ave Salinas Hicks, Clair & Linda CA 93901 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 2 2 2 2 2 2 2 Dues Assessment Dues Assessment Dues Dues Dues 2 1 96 350 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 28 larryshirlee@sbcglobal.net, w.d.hull@hotmail.com Hull, Wayne & Donna 65 300 300 200 100 300 400 100 200 250 1308 1310 1779 2115 2232 2250 9765 dues assessment dues buoy field assessment Dues - partial Dues - partial dues 7/21/2008 8/21/2008 6/8/2009 3/10/2010 6/14/2010 6/14/2010 9/12/2011 46620 Quail Run Lane Indian Wells Hull, Wayne & Donna CA 92210 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 28 28 28 28 28 28 28 Dues Assessment Dues Assessment Dues Dues Dues 28 1 65 900 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 8 jack520@pacbell.net Isberner, Jack & Corinne 72 300 300 300 -300 -5.00 500 200 300 1052381478 1052381478 1069899783 1081718785 1081676928 dues & assessment check bounced bounce fee from bank dues dues & buoy field assessment dues & buoy field assessment 2/18/2009 2/18/2009 2/18/2009 8/4/2009 3/19/2010 3/19/2010 1220 Manning Dr. El Dorado Hills Isberner, Jack & Corinne CA 95762 200 100 300 -5 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Moana covering bounce fee - held check too long Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 6/29/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 8 8 8 8 8 8 8 8 Dues Assessment Dues Correction Assessment Dues Dues Dues 8 1 72 300 16150 5 usps Assessment reminder & past-due statement printed 2/8/2010 Assessment reminder & past-due statement mailed 2/10/2010 Dues letter sent 8/2/2010 Statement printed 4/4/2011 Statement sent 4/5/2011 Statement sent 5/1/2012 10 janaluchini@securitasinc.com, Benicia518@yahoo.com Luchini & Pygeorge 64 300 300 200 100 300 400 300 300 1235 1260 1295 1327 1377 1418 dues assessment dues buoy field assessment Dues Dues 8/21/2008 12/29/2008 6/8/2009 12/2/2009 9/30/2010 5/24/2011 912 Hawthorne Dr. Rodeo Luchini & Pygeorge CA 94572 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 10 10 10 10 10 10 10 Dues Assessment Dues Assessment Dues Dues Dues 10 1 64 300 16150 5 email Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 1 phyllmates@aol.com Matesso, Russ & Phyliss 100 300 300 200 100 300 400 300 300 4011 4040 4443 4678 5032 5372 dues assessment dues buoy field assessment Dues Dues 8/21/2008 8/21/2008 6/8/2009 12/2/2009 9/30/2010 7/13/2011 1883 Cresmont Dr. San Jose Matesso, Russ & Phyliss CA 95124 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 1 1 1 1 1 1 1 Dues Assessment Dues Assessment Dues Dues Dues 1 1 100 1200 16150 5 usps Assessment reminder & past-due statement printed 2/8/2010 Assessment reminder & past-due statement mailed 2/10/2010 Dues letter sent 8/2/2010 Statement printed 4/4/2011 Statement sent 4/5/2011 Statement sent 5/1/2012 5 6 marcialeemiller@yahoo.com Miller, Marci 80 x 300 600 200 -200 -5.00 605 700 1300 851 851 1165 1388 1925 dues check bounced bounce fee from bank dues buoy field assessment & dues dues & assessment 8/21/2008 8/25/2008 8/25/2008 7/23/2009 3/19/2010 9/12/2011 5320 Divot Circle Fair Oaks Miller, Marci CA 95628 200 200 100 100 300 300 400 400 300 300 300 300 300 300 Dues 2008-2009 Dues 2008-2009 Assessment 2008-2009 pier ladders Assessment 2008-2009 pier ladders Dues 2009-2010 Dues 2009-2010 Fall 2009 buoy field assessment Fall 2009 buoy field assessment Dues 2010-2011 Dues 2010-2011 Dues 2011 Dues 2011 Dues 2012 Dues 2012 7/19/2008 7/19/2008 7/19/2008 7/19/2008 6/8/2009 6/8/2009 10/14/2009 10/14/2009 7/26/2010 7/26/2010 4/1/2011 4/1/2011 4/30/2012 4/30/2012 5 6 5 6 5 6 5 6 5 6 5 6 5 6 Dues Dues Assessment Assessment Dues Dues Assessment Assessment Dues Dues Dues Dues Dues Dues 5 6 1 1 80 x 300 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 11 toniharsh@charter.net Mollett Family 56 300 300 300 300 400 300 300 3932 1508 1562 1594 1666 dues & assessment dues buoy field assessment Dues Dues 8/21/2008 9/28/2009 3/10/2010 6/14/2010 5/24/2011 POBox 411 890 Marsh Ave. Homewood Reno Mollett Family Toni Harsh CA NV 96141 89509 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 11 11 11 11 11 11 11 Dues Assessment Dues Assessment Dues Dues Dues 11 1 56 450 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 12 jmozart@mozartdev.com Mozart, John & Heather 52 300 450 300 150 450 600 450 450 15152 15687 16413 17348 17988 18892 dues assessment dues buoy field assessment Dues Dues 7/21/2008 12/29/2008 7/23/2009 3/19/2010 9/30/2010 5/24/2011 1068 East Meadow Cr. Palo Alto Mozart, John & Heather CA 94303 300 150 450 600 450 450 450 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 12 12 12 12 12 12 12 Dues Assessment Dues Assessment Dues Dues Dues 12 1.5 52 600 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 23 26 pypy@sbcglobal.net Pygeorge, Michael & Janet 8 85 300 600 200 200 200 300 300 800 600 600 7483 7482 7692 7771 7770 8022 8159 8315 dues dues assessment dues dues buoy field assessment Dues Dues 7/21/2008 7/21/2008 5/7/2009 6/8/2009 6/8/2009 3/10/2010 8/14/2010 5/24/2011 512 Barnes Way Rodeo Pygeorge, Michael & Janet CA 94572 200 200 100 100 300 300 400 400 300 300 300 300 300 300 Dues 2008-2009 Dues 2008-2009 Assessment 2008-2009 pier ladders Assessment 2008-2009 pier ladders Dues 2009-2010 Dues 2009-2010 Fall 2009 buoy field assessment Fall 2009 buoy field assessment Dues 2010-2011 Dues 2010-2011 Dues 2011 Dues 2011 Dues 2012 Dues 2012 7/19/2008 7/19/2008 7/19/2008 7/19/2008 6/8/2009 6/8/2009 10/14/2009 10/14/2009 7/26/2010 7/26/2010 4/1/2011 4/1/2011 4/30/2012 4/30/2012 23 26 23 26 23 26 23 26 23 26 23 26 23 26 Dues Dues Assessment Assessment Dues Dues Assessment Assessment Dues Dues Dues Dues Dues Dues 23 26 1 1 8 85 300 16150 5 email Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement resent 4/7/2011 Statement sent 5/1/2012 3 daagbf@comcast.net Richardson, Don & Marilyn 92 300 300 200 100 300 400 300 300 13434 13440 13826 13997 14298 14585 dues assessment dues buoy field assessment Dues Dues 7/21/2008 8/21/2008 6/8/2009 11/24/2009 8/14/2010 5/24/2011 18695 Montewood Dr. Saratoga Richardson, Don & Marilyn CA 95070 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 3 3 3 3 3 3 3 Dues Assessment Dues Assessment Dues Dues Dues 3 1 92 300 16150 5 email Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 19 wthrelfall@pacbell.net Threlfall, Bill & Sandy 24 300 300 200 100 300 400 300 300 4499 4516 4553 4572 4615 4661 dues dues buoy field assessment Dues Dues 7/21/2008 12/29/2008 6/8/2009 11/10/2009 7/15/2010 5/24/2011 11 Woodside Glen Court Oakland Threlfall, Bill & Sandy CA 94602 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 19 19 19 19 19 19 19 Dues Assessment Dues Assessment Dues Dues Dues 19 1 24 1950 16150 5 email Assessment reminder & past-due statement emailed 2/10/2010 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 14 donoptions@aol.com Wells, Don, Kathy, & Debbie 48 300 450 650 550 1966 2498 dues dues 6/8/2009 7/23/2009 1133 Shoreline Court Copperopolis Wells, Don, Kathy, & Debbie CA 95228 300 300 150 450 600 450 450 450 2007-08 dues late Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 14 14 14 14 14 14 14 14 begin Dues Assessment Dues Assessment Dues Dues Dues 14 1.5 48 300 16150 5 email 0 Dues statement sent 8/2/2010 Statement sent 4/6/2011 Statement sent 5/1/2012 21 frankwin@gmail.com Winiarski, Frank 16 300 300 200 100 300 400 300 300 1271 1326 1362 1414 1478 1493 dues assessment dues buoy field assessment Dues Dues 7/21/2008 12/29/2008 6/25/2009 11/24/2009 8/14/2010 5/24/2011 200 West 2nd Street #1207 Reno Winiarski, Frank NV 89501 200 100 300 400 300 300 300 Dues 2008-2009 Assessment 2008-2009 pier ladders Dues 2009-2010 Fall 2009 buoy field assessment Dues 2010-2011 Dues 2011 Dues 2012 7/19/2008 7/19/2008 6/8/2009 10/14/2009 7/26/2010 4/1/2011 4/30/2012 21 21 21 21 21 21 21 Dues Assessment Dues Assessment Dues Dues Dues 21 1 16 \ No newline at end of file diff --git a/spec/data/db_fmpxmlresult.xml b/spec/data/db_fmpxmlresult.xml new file mode 100644 index 00000000..32b346cd --- /dev/null +++ b/spec/data/db_fmpxmlresult.xml @@ -0,0 +1,41 @@ + + + 0 + + + + + + + + + FMServer_Sample + + + + + Graphene + + + + + LucyLuLu + + + + + Palais + + + + + PalaisWeb + + + + + Totel + + + + diff --git a/spec/data/layout.xml b/spec/data/layout.xml new file mode 100644 index 00000000..a032a8e3 --- /dev/null +++ b/spec/data/layout.xml @@ -0,0 +1,330 @@ + + + + 0 + + + +