Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

blink1073
Copy link
Collaborator

Bottom line: array passing is 10X faster without using temporary files.
Uses base64 encoding to pass arrays between Matlab and Python using a similar method to complex. For Python, numeric-type arrays are converted to float64 and then passed as a dictionary. For Matlab, numeric arrays are converted to double and sent back as a structure.
Also handles complex type arrays. Side benefit: arrays are now returned as ndarrays.

With Base64:
In [3]: %time mlab.set_variable('ha', np.linspace(1.1, 2, 10000).reshape(100, 100))
CPU times: user 4.31 ms, sys: 1.61 ms, total: 5.92 ms
Wall time: 163 ms
Out[3]: {'success': 'true', 'result': 1, 'message': 'Successfully completed request'}

In [4]: %time b = mlab.get_variable('ha')
CPU times: user 3.93 ms, sys: 1.09 ms, total: 5.02 ms
Wall time: 66.8 ms

In [5]: b
Out[5]: b
array([[ 1.1       ,  1.10009001,  1.10018002, ...,  1.10873087,
         1.10882088,  1.10891089],
       [ 1.1090009 ,  1.10909091,  1.10918092, ...,  1.11773177,
         1.11782178,  1.11791179],
       [ 1.1180018 ,  1.11809181,  1.11818182, ...,  1.12673267,
         1.12682268,  1.12691269],
       ..., 
       [ 1.97308731,  1.97317732,  1.97326733, ...,  1.98181818,
         1.98190819,  1.9819982 ],
       [ 1.98208821,  1.98217822,  1.98226823, ...,  1.99081908,
         1.99090909,  1.9909991 ],
       [ 1.99108911,  1.99117912,  1.99126913, ...,  1.99981998,
         1.99990999,  2.        ]])

On master branch:
In [5]: %time mlab.set_variable('ha', np.linspace(1.1, 2, 10000).reshape(100, 100))
CPU times: user 16.7 ms, sys: 1.08 ms, total: 17.7 ms
Wall time: 1.19 s
Out[5]: {'result': 1, 'message': 'Successfully completed request', 'success': 'true'}

In [6]: %time b = mlab.get_variable('ha')
CPU times: user 5.33 ms, sys: 1.2 ms, total: 6.54 ms
Wall time: 821 ms

In [7]: type(b)
Out[7]: list

@blink1073
Copy link
Collaborator Author

I'm still cleaning this up/getting tests to pass.

@blink1073
Copy link
Collaborator Author

All tests pass!

@blink1073
Copy link
Collaborator Author

I'd prefer this to the create-a-file approach, since it is all in memory, and does not require you to be on the same machine to share the data.

@blink1073
Copy link
Collaborator Author

I'm working on support for complex arrays, don't merge yet.

@blink1073
Copy link
Collaborator Author

All set!

@arokem
Copy link
Owner

arokem commented Jan 17, 2015

Sounds great. I will not have time to look at it before Monday, but I will
do it then.

On Saturday, January 17, 2015, Steven Silvester [email protected]
wrote:

All set!


Reply to this email directly or view it on GitHub
#103 (comment)
.

@blink1073
Copy link
Collaborator Author

This was also an exercise in grokking the code base. FYI, my Matlab license expires in ~40 days.

%
% See also BASE64DECODE.

% Author: Peter J. Acklam
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think Peter is OK with us using his code? Did the original have a license attached to it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

http://www.fourmilab.ch/webtools/base64/

"This software is in the public domain. Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, without any conditions or restrictions. This software is provided “as is” without express or implied warranty. "

@blink1073
Copy link
Collaborator Author

Go for it.

arokem added a commit that referenced this pull request Jan 20, 2015
@arokem arokem merged commit 47ac7c6 into arokem:master Jan 20, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants