Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit ad00805

Browse files
gto76gto76
committed
Rewrite of Profiling
1 parent 0248487 commit ad00805

File tree

5 files changed

+109
-94
lines changed

5 files changed

+109
-94
lines changed

‎README.md

Lines changed: 37 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1935,14 +1935,14 @@ with <conn>.begin(): ... # Exits the block with commit or
19351935
```
19361936

19371937
```text
1938-
+------------+--------------+-----------+----------------------------------+
1939-
| Dialect | pip3 install | import | Dependencies |
1940-
+------------+--------------+-----------+----------------------------------+
1941-
| mysql | mysqlclient | MySQLdb | www.pypi.org/project/mysqlclient |
1942-
| postgresql | psycopg2 | psycopg2 | www.pypi.org/project/psycopg2 |
1943-
| mssql | pyodbc | pyodbc | www.pypi.org/project/pyodbc |
1944-
| oracle | oracledb | oracledb | www.pypi.org/project/oracledb |
1945-
+------------+--------------+-----------+----------------------------------+
1938+
+------------+--------------+----------+----------------------------------+
1939+
| Dialect | pip3 install | import | Dependencies |
1940+
+------------+--------------+----------+----------------------------------+
1941+
| mysql | mysqlclient | MySQLdb | www.pypi.org/project/mysqlclient |
1942+
| postgresql | psycopg2 | psycopg2 | www.pypi.org/project/psycopg2 |
1943+
| mssql | pyodbc | pyodbc | www.pypi.org/project/pyodbc |
1944+
| oracle | oracledb | oracledb | www.pypi.org/project/oracledb |
1945+
+------------+--------------+----------+----------------------------------+
19461946
```
19471947

19481948

@@ -2591,49 +2591,48 @@ duration_in_seconds = perf_counter() - start_time
25912591
### Timing a Snippet
25922592
```python
25932593
>>> from timeit import timeit
2594-
>>> timeit("''.join(str(i) for i in range(100))",
2595-
... number=10000, globals=globals(), setup='pass')
2596-
0.34986
2594+
>>> timeit('list(range(10000))', number=1000, globals=globals(), setup='pass')
2595+
0.19373
25972596
```
25982597

25992598
### Profiling by Line
2600-
```python
2601-
# $ pip3 install line_profiler memory_profiler
2599+
```text
2600+
$ pip3 install line_profiler
2601+
$ echo "
26022602
@profile
26032603
def main():
2604-
a = [*range(10000)]
2605-
b = {*range(10000)}
2606-
main()
2607-
```
2608-
2609-
```text
2604+
a = list(range(10000))
2605+
b = set(range(10000))
2606+
main()" > test.py
26102607
$ kernprof -lv test.py
26112608
Line # Hits Time Per Hit % Time Line Contents
26122609
=======================================================
26132610
1 @profile
26142611
2 def main():
2615-
3 1 955.0 955.0 43.7 a = [*range(10000)]
2616-
4 1 1231.0 1231.0 56.3 b = {*range(10000)}
2617-
2618-
$ python3 -m memory_profiler test.py
2619-
Line # Mem usage Increment Line Contents
2620-
=======================================================
2621-
1 37.668 MiB 37.668 MiB @profile
2622-
2 def main():
2623-
3 38.012 MiB 0.344 MiB a = [*range(10000)]
2624-
4 38.477 MiB 0.465 MiB b = {*range(10000)}
2612+
3 1 219.0 219.0 31.1 a = list(range(10000))
2613+
4 1 487.0 487.0 68.9 b = set(range(10000))
26252614
```
26262615

2627-
### Call Graph
2628-
#### Generates a PNG image of the call graph with highlighted bottlenecks:
2629-
```python
2630-
# $ pip3 install pycallgraph2; apt/brew install graphviz
2631-
import pycallgraph2 as cg, datetime
2616+
### Call and Flame Graphs
2617+
```bash
2618+
$ pip3 install gprof2dot snakeviz
2619+
$ apt/brew install graphviz
2620+
$ python3 -m cProfile -o test.prof test.py
2621+
$ gprof2dot -f pstats test.prof | dot -Tpng -o test.png; xdg-open/open test.png
2622+
$ snakeviz test.prof
2623+
```
26322624

2633-
filename = f'profile-{datetime.datetime.now():%Y%m%d_%H%M%S}.png'
2634-
drawer = cg.output.GraphvizOutput(output_file=filename)
2635-
with cg.PyCallGraph(drawer):
2636-
<code_to_be_profiled>
2625+
### Sampling and Memory Profilers
2626+
```text
2627+
+--------------+-------------------------------+------------+----------+------+
2628+
| pip3 install | How to run | Target | Type | Live |
2629+
+--------------+-------------------------------+------------+----------+------+
2630+
| py-spy | py-spy top -- python3 test.py | CPU | Sampling | Yes |
2631+
| pyinstrument | pyinstrument test.py | CPU | Sampling | No |
2632+
| scalene | scalene test.py | CPU+Memory | Sampling | No |
2633+
| memray | memray run --live test.py | Memory | Tracing | Yes |
2634+
| filprofiler | fil-profile run test.py | Memory | Tracing | No |
2635+
+--------------+-------------------------------+------------+----------+------+
26372636
```
26382637

26392638

‎index.html

Lines changed: 41 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@
5454

5555
<body>
5656
<header>
57-
<aside>June 9, 2023</aside>
57+
<aside>June 30, 2023</aside>
5858
<a href="https://gto76.github.io" rel="author">Jure Šorn</a>
5959
</header>
6060

@@ -1609,14 +1609,14 @@
16091609
<span class="hljs-keyword">with</span> &lt;conn&gt;.begin(): ... <span class="hljs-comment"># Exits the block with commit or rollback.</span>
16101610
</code></pre></div>
16111611

1612-
<pre><code class="text language-text">┏━━━━━━━━━━━━┯━━━━━━━━━━━━━━┯━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
1613-
┃ Dialect │ pip3 install │ import │ Dependencies ┃
1614-
┠────────────┼──────────────┼───────────┼──────────────────────────────────┨
1615-
┃ mysql │ mysqlclient │ MySQLdb │ www.pypi.org/project/mysqlclient ┃
1616-
┃ postgresql │ psycopg2 │ psycopg2 │ www.pypi.org/project/psycopg2 ┃
1617-
┃ mssql │ pyodbc │ pyodbc │ www.pypi.org/project/pyodbc ┃
1618-
┃ oracle │ oracledb │ oracledb │ www.pypi.org/project/oracledb ┃
1619-
┗━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
1612+
<pre><code class="text language-text">┏━━━━━━━━━━━━┯━━━━━━━━━━━━━━┯━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
1613+
┃ Dialect │ pip3 install │ import │ Dependencies ┃
1614+
┠────────────┼──────────────┼──────────┼──────────────────────────────────┨
1615+
┃ mysql │ mysqlclient │ MySQLdb │ www.pypi.org/project/mysqlclient ┃
1616+
┃ postgresql │ psycopg2 │ psycopg2 │ www.pypi.org/project/psycopg2 ┃
1617+
┃ mssql │ pyodbc │ pyodbc │ www.pypi.org/project/pyodbc ┃
1618+
┃ oracle │ oracledb │ oracledb │ www.pypi.org/project/oracledb ┃
1619+
┗━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
16201620
</code></pre>
16211621
<div><h2 id="bytes"><a href="#bytes" name="bytes">#</a>Bytes</h2><p><strong>Bytes object is an immutable sequence of single bytes. Mutable version is called bytearray.</strong></p><pre><code class="python language-python hljs">&lt;bytes&gt; = <span class="hljs-string">b'&lt;str&gt;'</span> <span class="hljs-comment"># Only accepts ASCII characters and \x00-\xff.</span>
16221622
&lt;int&gt; = &lt;bytes&gt;[&lt;index&gt;] <span class="hljs-comment"># Returns an int in range from 0 to 255.</span>
@@ -2124,44 +2124,43 @@ <h3 id="format-2">Format</h3><div><h4 id="forstandardtypesizesandmanualalignment
21242124
</code></pre></div>
21252125

21262126
<div><h3 id="timingasnippet">Timing a Snippet</h3><pre><code class="python language-python hljs"><span class="hljs-meta">&gt;&gt;&gt; </span><span class="hljs-keyword">from</span> timeit <span class="hljs-keyword">import</span> timeit
2127-
<span class="hljs-meta">&gt;&gt;&gt; </span>timeit(<span class="hljs-string">"''.join(str(i) for i in range(100))"</span>,
2128-
<span class="hljs-meta">... </span> number=<span class="hljs-number">10000</span>, globals=globals(), setup=<span class="hljs-string">'pass'</span>)
2129-
<span class="hljs-number">0.34986</span>
2130-
</code></pre></div>
2131-
2132-
<div><h3 id="profilingbyline">Profiling by Line</h3><pre><code class="python language-python hljs"><span class="hljs-comment"># $ pip3 install line_profiler memory_profiler</span>
2133-
<span class="hljs-meta">@profile</span>
2134-
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span><span class="hljs-params">()</span>:</span>
2135-
a = [*range(<span class="hljs-number">10000</span>)]
2136-
b = {*range(<span class="hljs-number">10000</span>)}
2137-
main()
2138-
</code></pre></div>
2139-
2140-
<pre><code class="text language-text">$ kernprof -lv test.py
2127+
<span class="hljs-meta">&gt;&gt;&gt; </span>timeit(<span class="hljs-string">'list(range(10000))'</span>, number=<span class="hljs-number">1000</span>, globals=globals(), setup=<span class="hljs-string">'pass'</span>)
2128+
<span class="hljs-number">0.19373</span>
2129+
</code></pre></div>
2130+
2131+
<div><h3 id="profilingbyline">Profiling by Line</h3><pre><code class="text language-text">$ pip3 install line_profiler
2132+
$ echo "
2133+
@profile
2134+
def main():
2135+
a = list(range(10000))
2136+
b = set(range(10000))
2137+
main()" &gt; test.py
2138+
$ kernprof -lv test.py
21412139
Line # Hits Time Per Hit % Time Line Contents
21422140
=======================================================
21432141
1 @profile
21442142
2 def main():
2145-
3 1 955.0 955.0 43.7 a = [*range(10000)]
2146-
4 1 1231.0 1231.0 56.3 b = {*range(10000)}
2147-
2148-
$ python3 -m memory_profiler test.py
2149-
Line # Mem usage Increment Line Contents
2150-
=======================================================
2151-
1 37.668 MiB 37.668 MiB @profile
2152-
2 def main():
2153-
3 38.012 MiB 0.344 MiB a = [*range(10000)]
2154-
4 38.477 MiB 0.465 MiB b = {*range(10000)}
2155-
</code></pre>
2156-
<div><h3 id="callgraph">Call Graph</h3><div><h4 id="generatesapngimageofthecallgraphwithhighlightedbottlenecks">Generates a PNG image of the call graph with highlighted bottlenecks:</h4><pre><code class="python language-python hljs"><span class="hljs-comment"># $ pip3 install pycallgraph2; apt/brew install graphviz</span>
2157-
<span class="hljs-keyword">import</span> pycallgraph2 <span class="hljs-keyword">as</span> cg, datetime
2143+
3 1 219.0 219.0 31.1 a = list(range(10000))
2144+
4 1 487.0 487.0 68.9 b = set(range(10000))
2145+
</code></pre></div>
21582146

2159-
filename = <span class="hljs-string">f'profile-<span class="hljs-subst">{datetime.datetime.now():%Y%m%d_%H%M%S}</span>.png'</span>
2160-
drawer = cg.output.GraphvizOutput(output_file=filename)
2161-
<span class="hljs-keyword">with</span> cg.PyCallGraph(drawer):
2162-
&lt;code_to_be_profiled&gt;
2163-
</code></pre></div></div>
2147+
<div><h3 id="callandflamegraphs">Call and Flame Graphs</h3><pre><code class="bash language-bash hljs">$ pip3 install gprof2dot snakeviz
2148+
$ apt/brew install graphviz
2149+
$ python3 -m cProfile -o test.prof test.py
2150+
$ gprof2dot -f pstats test.prof | dot -Tpng -o test.png; xdg-open/open test.png
2151+
$ snakeviz test.prof
2152+
</code></pre></div>
21642153

2154+
<div><h3 id="samplingandmemoryprofilers">Sampling and Memory Profilers</h3><pre><code class="text language-text">┏━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━┯━━━━━━┓
2155+
┃ pip3 install │ How to run │ Target │ Type │ Live ┃
2156+
┠──────────────┼───────────────────────────────┼────────────┼──────────┼──────┨
2157+
┃ py-spy │ py-spy top -- python3 test.py │ CPU │ Sampling │ Yes ┃
2158+
┃ pyinstrument │ pyinstrument test.py │ CPU │ Sampling │ No ┃
2159+
┃ scalene │ scalene test.py │ CPU+Memory │ Sampling │ No ┃
2160+
┃ memray │ memray run --live test.py │ Memory │ Tracing │ Yes ┃
2161+
┃ filprofiler │ fil-profile run test.py │ Memory │ Tracing │ No ┃
2162+
┗━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━┷━━━━━━┛
2163+
</code></pre></div>
21652164

21662165
<div><h2 id="numpy"><a href="#numpy" name="numpy">#</a>NumPy</h2><p><strong>Array manipulation mini-language. It can run up to one hundred times faster than the equivalent Python code. An even faster alternative that runs on a GPU is called CuPy.</strong></p><pre><code class="python language-python hljs"><span class="hljs-comment"># $ pip3 install numpy</span>
21672166
<span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np
@@ -2934,7 +2933,7 @@ <h3 id="format-2">Format</h3><div><h4 id="forstandardtypesizesandmanualalignment
29342933

29352934

29362935
<footer>
2937-
<aside>June 9, 2023</aside>
2936+
<aside>June 30, 2023</aside>
29382937
<a href="https://gto76.github.io" rel="author">Jure Šorn</a>
29392938
</footer>
29402939

‎parse.js

Lines changed: 28 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -460,19 +460,19 @@ const DIAGRAM_9_B =
460460
"┗━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┛\n";
461461

462462
const DIAGRAM_95_A =
463-
"+------------+--------------+-----------+----------------------------------+\n" +
464-
"| Dialect | pip3 install | import | Dependencies |\n" +
465-
"+------------+--------------+-----------+----------------------------------+\n";
463+
'+------------+--------------+----------+----------------------------------+\n' +
464+
'| Dialect | pip3 install | import | Dependencies |\n' +
465+
'+------------+--------------+----------+----------------------------------+\n';
466466

467467
const DIAGRAM_95_B =
468-
"┏━━━━━━━━━━━━┯━━━━━━━━━━━━━━┯━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n" +
469-
"┃ Dialect │ pip3 install │ import │ Dependencies ┃\n" +
470-
"┠────────────┼──────────────┼───────────┼──────────────────────────────────┨\n" +
471-
"┃ mysql │ mysqlclient │ MySQLdb │ www.pypi.org/project/mysqlclient ┃\n" +
472-
"┃ postgresql │ psycopg2 │ psycopg2 │ www.pypi.org/project/psycopg2 ┃\n" +
473-
"┃ mssql │ pyodbc │ pyodbc │ www.pypi.org/project/pyodbc ┃\n" +
474-
"┃ oracle │ oracledb │ oracledb │ www.pypi.org/project/oracledb ┃\n" +
475-
"┗━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛\n";
468+
'┏━━━━━━━━━━━━┯━━━━━━━━━━━━━━┯━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n' +
469+
'┃ Dialect │ pip3 install │ import │ Dependencies ┃\n' +
470+
'┠────────────┼──────────────┼──────────┼──────────────────────────────────┨\n' +
471+
'┃ mysql │ mysqlclient │ MySQLdb │ www.pypi.org/project/mysqlclient ┃\n' +
472+
'┃ postgresql │ psycopg2 │ psycopg2 │ www.pypi.org/project/psycopg2 ┃\n' +
473+
'┃ mssql │ pyodbc │ pyodbc │ www.pypi.org/project/pyodbc ┃\n' +
474+
'┃ oracle │ oracledb │ oracledb │ www.pypi.org/project/oracledb ┃\n' +
475+
'┗━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛\n';
476476

477477
const DIAGRAM_10_A =
478478
'+-------------+-------------+\n' +
@@ -508,6 +508,22 @@ const DIAGRAM_11_B =
508508
'┃ str │ ┃\n' +
509509
'┗━━━━━━━━━━━━━┷━━━━━━━━━━━━━┛\n';
510510

511+
const DIAGRAM_115_A =
512+
'+--------------+-------------------------------+------------+----------+------+\n' +
513+
'| pip3 install | How to run | Target | Type | Live |\n' +
514+
'+--------------+-------------------------------+------------+----------+------+\n';
515+
516+
const DIAGRAM_115_B =
517+
'┏━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━┯━━━━━━┓\n' +
518+
'┃ pip3 install │ How to run │ Target │ Type │ Live ┃\n' +
519+
'┠──────────────┼───────────────────────────────┼────────────┼──────────┼──────┨\n' +
520+
'┃ py-spy │ py-spy top -- python3 test.py │ CPU │ Sampling │ Yes ┃\n' +
521+
'┃ pyinstrument │ pyinstrument test.py │ CPU │ Sampling │ No ┃\n' +
522+
'┃ scalene │ scalene test.py │ CPU+Memory │ Sampling │ No ┃\n' +
523+
'┃ memray │ memray run --live test.py │ Memory │ Tracing │ Yes ┃\n' +
524+
'┃ filprofiler │ fil-profile run test.py │ Memory │ Tracing │ No ┃\n' +
525+
'┗━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━┷━━━━━━┛\n';
526+
511527
const DIAGRAM_12_A =
512528
'+-----------+-----------+------+-----------+\n' +
513529
'| sampwidth | min | zero | max |\n' +
@@ -740,6 +756,7 @@ function updateDiagrams() {
740756
$(`code:contains(${DIAGRAM_95_A})`).html(DIAGRAM_95_B);
741757
$(`code:contains(${DIAGRAM_10_A})`).html(DIAGRAM_10_B);
742758
$(`code:contains(${DIAGRAM_11_A})`).html(DIAGRAM_11_B);
759+
$(`code:contains(${DIAGRAM_115_A})`).html(DIAGRAM_115_B);
743760
$(`code:contains(${DIAGRAM_12_A})`).html(DIAGRAM_12_B).removeClass("text").removeClass("language-text").addClass("python");
744761
$(`code:contains(${DIAGRAM_13_A})`).html(DIAGRAM_13_B).removeClass("text").removeClass("language-text").addClass("python");
745762
$(`code:contains(${DIAGRAM_14_A})`).parent().remove();

0 commit comments

Comments
 (0)