Hacking WebKit &
Its JavaScript Engines
Jarred Nicholls
Work @ Sencha
WebKit Committer
Doing webkitty things...
Our Mission if we choose to accept it
Define: What is WebKit?
Some Simple Ways to Leverage WebKit
Little investment, Large ROI
Web Inspector Hacking
“Headless” WebKit
What is WebKit?
≠
Browser at a High Level
User Interface
Data Persistence
Browser Engine
Render Engine
Networking JavaScript Graphics
I/O Engine Stack
Browser at a High Level
User Interface
We
Data Persistence
bK
it Browser Engine
Render Engine
Networking JavaScript Graphics
I/O Engine Stack
≠
⊂
WebKit is
EVERYWHERE!
Even in your living room!
Portability
The WebKit project seeks to address a variety of needs. We want to
make it reasonable to port WebKit to a variety of desktop,
mobile, embedded and other platforms. We will provide the
infrastructure to do this with tight platform integration, reusing native
platform services where appropriate and providing friendly
embedding APIs.
http://www.webkit.org/projects/goals.html
How?
Client Interface
WebKit Components
Render Engine
CSS
DOM SVG
HTML Canvas
WebCore
Client Interface JavaScript Engine (JSC/V8)
WebKit Components
Render Engine
CSS
DOM SVG
HTML Canvas
WebCore
Client Interface JavaScript Engine (JSC/V8)
Port Abstraction
Networking
Thread Geolocation Timer
I/O
Client Interface
Clipboard Events Theme Graphics
API Calls
Events
Port
(Chrome, Safari, etc.)
Port Abstraction
Networking
Thread Geolocation Timer
I/O
Client Interface
Clipboard Events Theme Graphics
API Calls
Events
Port
(Chrome, Safari, etc.)
<input type=”number” />
Paint me
some spin
buttons, STAT!
I know how to
do that!
WebKit Port
<input type=”number” />
ri
pl ght
ea th
se er
,k e
th
x
WebKit
<input type=”number” />
Piece of cake!
y !
ex
’ s
in
ok
lo Port
<input type=”number” />
Theme Interface
bool paintInnerSpinButton(RenderObject*, const
PaintInfo&, const IntRect&);
<input type=”number” />
Theme Interface
bool paintInnerSpinButton(RenderObject*, const
PaintInfo&, const IntRect&);
Theme
Client Interface
RenderTheme->paint()
paintInnerSpinButton Port
WebKit Components
Render Engine
CSS
DOM SVG
HTML Canvas
WebCore
Client Interface JavaScript Engine (JSC/V8)
JavaScript Bindings
JavaScript Bindings
ArrayBuffer.idl
Web IDL
gen
erate
- bi n
ding
s.pl
ArrayBuffer
Wrapper
JavaScript Engine (JSC/V8)
ArrayBuffer
WebCore
ArrayBuffer.h
class
ArrayBuffer
:
public
RefCounted<ArrayBuffer>
{
public:
//
...
inline
unsigned
byteLength()
const;
inline
PassRefPtr<ArrayBuffer>
slice(int
begin,
int
end)
const;
inline
PassRefPtr<ArrayBuffer>
slice(int
begin)
const;
//
...
};
Web[Kit]IDL
interface
[
JSGenerateIsReachable=Impl,
CustomConstructor,
ConstructorParameters=1,
JSNoStaticTables
]
ArrayBuffer
{
readonly
attribute
int
byteLength;
ArrayBuffer
slice(in
long
begin,
in
[Optional]
long
end);
};
>
perl
generate-‐bindings.pl
-‐-‐generator
JS
ArrayBuffer.idl
>
perl
generate-‐bindings.pl
-‐-‐generator
V8
ArrayBuffer.idl
JavaScriptCore
JSValue
jsArrayBufferByteLength(ExecState*
exec,
JSValue
slotBase,
PropertyName)
{
JSArrayBuffer*
castedThis
=
jsCast<JSArrayBuffer*>(asObject(slotBase));
UNUSED_PARAM(exec);
ArrayBuffer*
impl
=
static_cast<ArrayBuffer*>(castedThis-‐>impl());
JSValue
result
=
jsNumber(impl-‐>byteLength());
return
result;
}
V8
static
v8::Handle<v8::Value>
byteLengthAttrGetter(v8::Local<v8::String>
name,
const
v8::AccessorInfo&
info)
{
INC_STATS("DOM.ArrayBuffer.byteLength._get");
ArrayBuffer*
imp
=
V8ArrayBuffer::toNative(info.Holder());
return
v8::Integer::New(imp-‐>byteLength());
}
Hacking WebKit
For Fun & Profit
Building WebKit Nightly
Prerequisites
Prerequisites
Mac
Xcode 3.1.4+ & command-line tools
> which g++ => /usr/bin/g++
Windows
Don’t even bother (see “Building Chromium” :)
...but if you’re really brave: http://www.webkit.org/building/tools.html
Get the Source
Get the Source
SVN
svn checkout http://svn.webkit.org/repository/webkit/trunk WebKit
Git (RECOMMENDED!!!)
git clone git://git.webkit.org/WebKit.git
or
git clone git://github.com/WebKit/webkit.git WebKit
Build WebKit
Build WebKit
> cd WebKit
> Tools/Scripts/build-webkit
Go have lunch...
> Tools/Scripts/run-safari
Build WebKit
> cd WebKit
> Tools/Scripts/build-webkit
Go have lunch... runs local safari, but w/ new
WebKit framework
> Tools/Scripts/run-safari
Building Chromium
Prerequisites
Prerequisites
Depot Tools: http://www.chromium.org/developers/how-tos/install-depot-tools
Mac
Xcode 4.3+ & command-line tools
> which clang => /usr/bin/clang
Windows
Visual Studio (Express)
Windows SDK
DirectX SDK
Windows Driver Development Kit
Details: http://www.chromium.org/developers/how-tos/build-instructions-windows#TOC-Prerequisite-software
Get the Source
Get the Source
> mkdir chromium && cd chromium
> gclient config http://src.chromium.org/chrome/trunk/src
> gclient sync
Details: http://dev.chromium.org/developers/how-tos/get-the-code
Build Chromium
Build Chromium
> cd chromium/src
> GYP_GENERATORS=ninja gclient runhooks --force
> ninja -C out/Release chrome
Go on vacation...
> open out/Release/Chromium.app
Build Chromium
> cd chromium/src
> GYP_GENERATORS=ninja gclient runhooks --force
> ninja -C out/Release chrome
super fast “make” equivalent
Go on vacation... comes with depot tools
> open out/Release/Chromium.app
Let’s Hack
Web Inspector
Add in Hacks
inspector.js
var
re
=
/command=([^&]+)/,
match
=
re.exec(location.href),
command; command handler
if
(!match)
{
return;
}
try
{
command
=
eval('('
+
decodeURIComponent(match[1])
+
')');
}
catch
(e)
{
return;
}
switch
(command.type)
{
default:
break;
}
Add in Hacks
inspector.js persist data
var
saveData
=
function(filePath,
data)
{
filePath
=
'/Users/jarred/qconny/'
+
filePath;
var
xhr
=
new
XMLHttpRequest();
xhr.onload
=
function()
{
alert('Data
saved
successfully
to
'
+
filePath);
};
xhr.onerror
=
function()
{
alert('Data
failed
to
save
to
'
+
filePath);
};
xhr.open('GET',
'/savedata');
xhr.setRequestHeader('DevTools-‐Data',
JSON.stringify(data));
xhr.setRequestHeader('DevTools-‐FilePath',
filePath);
xhr.send(null);
};
Add in Hacks
devtools_http_handler_impl.cc
void
DevToolsHttpHandlerImpl::OnHttpRequest(
int
connection_id,
const
net::HttpServerRequestInfo&
info)
{
//
...
if
(info.path.find("/savedata")
==
0)
{
BrowserThread::PostTask(
BrowserThread::FILE,
FROM_HERE,
base::Bind(&DevToolsHttpHandlerImpl::OnSaveDataRequestFILE,
this,
connection_id,
info));
return;
}
//
... persist data
}
Add in Hacks
devtools_http_handler_impl.cc
void
DevToolsHttpHandlerImpl::OnSaveDataRequestFILE(int
connection_id,
const
net::HttpServerRequestInfo&
info)
{
bool
success
=
false;
FILE*
fp
=
fopen(info.GetHeaderValue("DevTools-‐FilePath").c_str(),
"wb");
if
(fp)
{
std::string
data
=
info.GetHeaderValue("DevTools-‐Data");
fwrite(data.data(),
1,
data.size(),
fp);
fclose(fp);
success
=
true;
}
Send200(connection_id,
success
?
"true"
:
"false",
"application/json;
charset=UTF-‐8");
}
persist data
Web Inspector
Network
Export Network HAR Data
More info: http://www.softwareishard.com/blog/har-12-spec/
Add in Hacks
inspector.js exportHAR cmd
var
exportHAR
=
function(cmd)
{
//
Force
a
reload
of
the
page
to
grab
network
stats.
PageAgent.reload(true
/*
bypass
cache
*/);
setTimeout(function()
{
var
networkData
=
WebInspector.panels.network._networkLogView._getExportData();
saveData((cmd.filename
||
WebInspector.inspectedPageDomain)
+
'.har',
networkData);
},
cmd.timeout);
};
case
'exportHAR':
exportHAR(command);
break;
Add in Hacks
NetworkPanel.js
export data fn
_getExportData:
function()
{
return
{
log:
(new
WebInspector.HARLog(this._requests)).build()
};
}
Trigger Hack
Trigger Hack
http://localhost:9123/devtools/devtools.html?ws=localhost:9123/devtools/page/
3_1&command={type:'exportHAR',timeout:3000}
HAR Output
{"log":{"version":"1.2","creator":
{"name":"WebInspector","version":"536.5"},"pages":[],"entries":
[{"startedDateTime":"2012-‐06-‐19T18:02:16.852Z","time":8,"request":
{"method":"GET","url":"http://www.google.com/
blank.html","httpVersion":"HTTP/1.1","headers":[],"queryString":
[],"cookies":[],"headersSize":47,"bodySize":0},"response":
{"status":200,"statusText":"OK","httpVersion":"HTTP/1.1","headers":
[],"cookies":[],"content":{"size":0,"mimeType":"text/
html"},"redirectURL":"","headersSize":17,"bodySize":0},"cache":
{},"timings":{"blocked":
0,"dns":-‐1,"connect":-‐1,"send":-‐1,"wait":-‐1,"receive":0,"ssl":-‐1}},
HAR Output
http://www.softwareishard.com/har/viewer/
Demo
Export Timeline Data
Add in Hacks
inspector.js exportTimeline cmd
var
exportTimeline
=
function(cmd)
{
setTimeout(function()
{
//
Start
the
timeline
profiler.
WebInspector.panels.timeline._toggleTimelineButtonClicked();
//
Force
a
reload
of
the
page
to
grab
latest
timeline
stats.
PageAgent.reload(true
/*
bypass
cache
*/);
setTimeout(function()
{
//
Stop
the
timeline
profiler.
WebInspector.panels.timeline._toggleTimelineButtonClicked();
var
data
=
WebInspector.panels.timeline._model.getExportData();
saveData(cmd.filename
||
(WebInspector.inspectedPageDomain
+
'-‐
timeline.json'),
data);
},
cmd.timeout);
},
1000);
};
case
'exportTimeline':
exportTimeline(command);
break;
Add in Hacks
TimelineModel.js
export data fn
getExportData:
function()
{
var
records
=
[];
for
(var
i
=
0;
i
<
this._records.length;
i++)
{
records.push(this._records[i]);
}
return
records;
}
Trigger Hack
Trigger Hack
http://localhost:9123/devtools/devtools.html?ws=localhost:9123/devtools/page/
3_1&command={type:'exportTimeline',timeout:3000}
Demo
Command Line
WebKit
Demo
How’d you do that!?
Headless WebKit
“Headless”
Normal Browser
Paint
Layout Display
Headless
Paint
X
Layout Display
What’s in it for me?
What’s in it for me?
Automated “smoke” tests
Continuous integration
Pixel-perfect regression tests
Web scraping
...
PhantomJS
Headless WebKit
Based on QtWebKit port
Full Web Stack
JS, DOM, CSS3, Canvas, SVG, etc.
Command Line Interface
JavaScript APIs
License: BSD / Open Source
http://www.phantomjs.org
Demo
Wrap Up
Hacking WebKit
Automation
Page scraping
Behavior/input emulation
Web Inspector hijacking
Quality Assurance / Continuous Integration
Smoke tests
Visual/pixel regression tests
Unit tests
Performance or Load regression tests
Jarred Nicholls
@jarrednicholls
Thanks!
