diff --git a/_includes/head.html b/_includes/head.html index 1667c40..09ad291 100644 --- a/_includes/head.html +++ b/_includes/head.html @@ -38,38 +38,15 @@ - - - - - - - diff --git a/_layouts/default.html b/_layouts/default.html index 1b85e6d..afc0d00 100644 --- a/_layouts/default.html +++ b/_layouts/default.html @@ -9,7 +9,7 @@
- diff --git a/_posts/2016-04-01-memory-leaks-in-java-applications-introduction.md b/_posts/2016-04-01-memory-leaks-in-java-applications-introduction.md index 7890da4..d5a3680 100644 --- a/_posts/2016-04-01-memory-leaks-in-java-applications-introduction.md +++ b/_posts/2016-04-01-memory-leaks-in-java-applications-introduction.md @@ -8,7 +8,7 @@ keywords: java, performance While memory leaks in the Java language are different than in languages with manual memory allocation, they can still become challenges to developers - -especially when they are discovered in the late phases of the project. +especially when they are discovered in the late phases of the project. @@ -18,7 +18,7 @@ leak is > (...) a type of resource leak that occurs when a computer program > incorrectly manages memory allocations in such a way that memory which is no -> longer needed is not released. +> longer needed is not released. While we do not directly allocate memory in Java, we do it indirectly - every time an object is created, some memory is allocated for it. In order for the @@ -43,7 +43,7 @@ A somewhat better definition of a memory leak in Java, would be: > A memory leak happens when there is a reference to a short lived object from > a long-lived context (another object, static context, etc.), which causes > the memory usage of an application to increase over time, until it runs out -> of memory. +> of memory. # Difference Between Memory Leaks and Poor Memory Usage Memory leaks are buzz words. They are keywords that you probably want to use @@ -60,15 +60,15 @@ following: - a normal pattern in the lifecycle of the garbage collector - poor memory usage due to how the data is modelled -In Java applications, the heap memory is allocated during a Garbage collector +In Java applications, the heap memory is deallocated during a Garbage collector run. An object is eligible for garbage collection if, - it is not a GC root - it is not accessible from GC root references (direct or transitive) -> **What is a GC root?** +> **What is a GC root?** > GC roots are special objects that are not garbage collected and keep -> references to the objects used by the application flows. +> references to the objects used by the application flows. > > - **Class** - classes loaded by the system class loader > - **Thread** - live thread @@ -93,7 +93,7 @@ areas of memory that can be reclaimable - and they will be reclaimed at some later point. # Signs of a memory leak -While it's by not means exhaustive, the following list illustrates some +While it's by not means exhaustive, the following list illustrates some common signs of a memory leak: - the used memory steadily increases over a long period of time (e.g. days, @@ -115,4 +115,3 @@ VisualVM or even jConsole. For now, this post got a bit longer than I anticipated initially. As time permits, I will try to write a follow-up with some hints on how to approach memory leaks. - diff --git a/_posts/2016-04-15-memory-leaks-in-java-heap-analysisblog.md b/_posts/2016-04-15-memory-leaks-in-java-heap-analysisblog.md new file mode 100644 index 0000000..6f459a7 --- /dev/null +++ b/_posts/2016-04-15-memory-leaks-in-java-heap-analysisblog.md @@ -0,0 +1,227 @@ +--- +layout: post +title: Memory Leaks in Java - Heap Analysis +date: 2016-04-15 +keywords: java, performance +--- + +In the [previous post]({% post_url 2016-04-01-memory-leaks-in-java-applications-introduction %}) +we've went through what are memory leaks in Java and how they can manifest +themselves. In this post, I will go through more details on how to find memory +leaks and how to investigate memory issues. + + + +### Tools of Trade +This guide will focus on the following tools: + +- [VisualVM](https://visualvm.java.net/) or Eclipse MAT to take a heap dump. +VisualVM should also be available in most JDK distributions. +- [Eclipse MAT](https://eclipse.org/mat/) for heap analysis + +These tools are seen as "traditional" ones in the Java eco-system and should be +avialable in all environments and in wide range of JDKs. There are some other +options available, some of them which may provided extended tools - but they +either require new JDK versions or are not free. + +Other tools that you can try, + +- [Oracle Mission Control](http://www.oracle.com/technetwork/java/javaseproducts/mission-control/java-mission-control-1998576.html) +- [Your Kit](https://www.yourkit.com/) +- [JProfiler](https://www.ej-technologies.com/products/jprofiler/overview.html) + +### How to Take a Heap Dump? +The first step in analysing memory issues is to get a heap dump when the issue +occurs. Unfortunately this may not always be enough and most of the time it is +recommended to also have some steps to reproduce the issue that you're trying to +fix. This will allow you to take incremental heap dumps and see how the memory +evolves across time. + +There are multiple ways to take a heap dump, but I will focus on two of them. + +#### Using VisualVM +VisualVM is bundled most JDK versions and can be used for monitoring and +profiling. You can take a heap dump from a local process, or from a remote +process through JMX. + +> The next instructions assume that you have JMX enabled for your JVM process. +> To learn more about JMX and how to enable it, see +> [Oracle JMX Management Guide](https://docs.oracle.com/javase/1.5.0/docs/guide/management/agent.html) + +To take a heap dump from a local JVM process, + +1. Under the local Applications section (left side menu), select the JVM process + that you would like to analyse. +2. Select the "Monitor" tab. +3. Select the "Heap Dump" button. + +To take a heap dump from a remote JVM process, + +1. Right click on the "Remote" section. +2. Enter the host name of the machine where the JVM process is running and + select OK. If you're encountering issues, be sure to check also the "Advanced + Section". +3. Select the newly added host and choose "Add a new JMX connection". It is also + possible to use jstatd, but it's beyound the scope of this guide to go into + details. +4. Fill in the required details according to the JMX configuration that you're + using. +5. Select OK. + +From this point on, the process is the same as when you're monitoring a local +JVM process. Select the "Monitor" tab use the "Heap Dump" button. + +> **Note 1:** The heap dump will be saved on the remote machine and you will be +> prompted to enter the path. + +> **Note 2:** Make sure that you have enough disk space available. The disk +> space needed is slightly the same as the size of the heap that you're trying +> to dump, but in some cases it can also be higher, with around 10-20%. + +#### Using Eclipse Memory Analyzer +Eclipse Memory Analyzer can also be used to take heap dumps from local JVM +process. If this is your case, it should probably be more simpler than using +VisualVM (one less tool needed). + +#### Take a Heap Dump after an OutOfMemoryError +You can also configure your application to generate a heap dump after an +`OutOfMemoryError` by addding the following parameter when launching the JVM: +`-XX:+HeapDumpOnOutOfMemoryError`. + +### Heap Analysis +Once you have a heap dump available, you can open it in an analysing tool. In +this guide we will focus on Eclipse Memory Analyzer, but some instructions may +also be similar for other tools. + +The first step would be to open the heap dump in Eclipse MAT. Be warned though, +for larger heap dumps you will need: + +- Physical RAM installed on the machine roughly equal or higher than the size of + the heap dump. +- Increase the maximum heap size available for Eclipse MAT. This can be done by + editing the `eclipse.ini` file and adding a line similar to `-Xmx2048m`. +- Free disk space on the machine roughly equal to 100-150% of the heap dump + size. + +The first time when you open a heap dump, Eclipse MAT will build various index +files which may take a long time, depending on the heap dump size. + +#### Get Familiar with Your Data +I think this is one of the most important steps that you can take when analysing +a heap dump. Sometimes you may be able to find a memory leak easily, just by +looking at the default reports produced by Eclipse, but in the most complicated +cases this will not be possible. + +The real issue may lie somewhere deeper - it may not be a single instance that +uses more than 90% of your heap, instead most of the heap usage could be spread +around millions of instances, that you cannot really keep track of. + +This is where it helps a lot of you know your data model. In a lot of cases, you +should be able to determine the expected number of instances of a certain class +that should be loaded in memory. + +Let's say that you're investigating a memory issue related to the login process. +If you can determine the number of logged in users at the time and also the +number of active sessions, you can map these to the actual number of user +instances from the memory. Do you have a 1 to 1 mapping? If not, maybe that's +where the problem is. + +The rule of thumb is to try to understand the number of instances and check if +this is correct. + +#### Use the Eclipse MAT Reports +If you don't know what you're looking for, it's usually good idea to go through +the reports suggested by Eclipse MAT, but try to take them with a grain of salt. +As mentined in the previous post, not all memory issues are real memory leaks - +while the leak report may sound dramatic, it doesn't mean that you actually have +a memory leak. You should consider the entities listed there as suspects and +evaluate them one by one. + +##### Histogram +As described in the tooltip, the Histogram "lists number of instances per +class". It can be a useful indicator in some cases - for example if you notice +an unusually high number of instances of a certain class. + +##### Dominator Tree +As described in the [Dominator Tree](http://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.mat.ui.help%2Fconcepts%2Fdominatortree.html) +section from the Eclipse MAT help, + +> An object x dominates an object y if every path in the object graph from the +> start (or the root) node to y must go through x. +> +> The immediate dominator x of some object y is the dominator closest to the +> object y. +> +> A dominator tree is built out of the object graph. In the dominator tree each +> object is the immediate dominator of its children, so dependencies between the +> objects are easily identified. + +This can be a very good starting point in understanding the memory usage. + +The other reports may also provide interesting data, be sure to also check them +out. + +#### Do You Know What to Look For? +If you already know what you're looking for, it is quite simple to get the data +that you need. Eclipse MAT supports a query language (OQL - Object Query +Language) that allows you to query the memory details for specific objects. + +For example, to get a list of all instances of the User class, you could enter +the following query: + +``` sql +select * from com.mypackage.model.User +``` + +and select the "!" (exclamation mark) symbol. + +You will get back a list of all objects of the specific class. For each +instance, you will see the shallow and retained heap used by this instance. +According to the Oracle documentation, + +> *Shallow heap* is the memory consumed by one object. An object needs 32 or 64 +> bits (depending on the OS architecture) per reference, 4 bytes per Integer, 8 +> bytes per Long, etc. Depending on the heap dump format the size may be +> adjusted (e.g. aligned to 8, etc...) to model better the real consumption of +> the VM. +> +> *Retained set* of X is the set of objects which would be removed by GC when X +> is garbage collected. +> +> *Retained heap* of X is the sum of shallow sizes of all objects in the +> retained set of X, i.e. memory kept alive by X. + +If you go with this approach, you should be careful not to count the same memory +twice. If object A holds a reference to object B, the retained size of A *may* +also contain the retain size of B. It will not contain the retain size +of B if there are other references to B, except the ones from A. + +##### Other options +If you right click on one of the results form the OQL query, you get access to a +context menu with several options. I will try to go through the ones that I find +the most useful: + +*List Objects - with their incoming references* +It will show the list of objects that point to the currently selected one. + +*List Objects - with their outgoing references* +This one is quite straightforward - it will show you the references that point +outside of the currently selected object (i.e. it's instance fields). + +*Path to GC roots* +As the name says, it will show you the path to the GC roots. Most of the time, +you will probably want to also ignore all phantom/soft/weak references. + +*Immediate dominators* +It will show the immediate parent object that prevents this object from being +garbage collected. + +*Copy - OQL Query* +It will generate an OQL that returns this single object. This is useful if you +dig deeper through the hiearchy. + + +## Other Resources +* [10 Tips for Using the Eclipse Memory Analyzer](http://eclipsesource.com/blogs/2013/01/21/10-tips-for-using-the-eclipse-memory-analyzer/) +* [Eclipse MAT Help](http://help.eclipse.org/mars/topic/org.eclipse.mat.ui.help/welcome.html) + diff --git a/_posts/2016-05-31-visualvm-sampling.md b/_posts/2016-05-31-visualvm-sampling.md new file mode 100644 index 0000000..2bb99d7 --- /dev/null +++ b/_posts/2016-05-31-visualvm-sampling.md @@ -0,0 +1,125 @@ +--- +layout: post +title: VisualVM CPU Sampling +date: 2016-05-31 +keywords: java, performance, sampling, profiling +--- + +If you need to investigate CPU related issues, sampling provides an easy +mechanism for identifying bottlenecks, with minimal effects on the performance. + + + +### Sampling vs Profiling + +First of all, let's understand the difference between sampling and profiling, +which is a key prerequisite. + +#### Profiling +Profiling involves instrumenting the entire application code or only some +classes in order to provide runtime performance metrics to the profiler +application. Since this involves changes to the application code, which are +applied automatically by the profiler, it also means that there is a certain +performance impact and risk of affecting the existing functionality. + +The actual degree of the performance impact is hard to determine, but it can +become significant if CPU intensive sections are instrumented. + +Profiling is usually recommended for optimizing specific algorithms or when +you're interested in measuring the invocation counts. + +#### Sampling +Sampling on the other side works by periodically retrieving thread dumps from +the JVM. In this case, the performance impact is minor (and constant since the +thread dumps are retrieved using a fixed frequency) and there's no risk of +introducing side effects. This process is a lot less intrusive and can also be +performed quite reliably on remote applications (i.e. it could even be applied +to production instances). + +The main downside of CPU sampling is the accuracy - since the thread dump is +retrieved at fixed intervals, there is a high risk of missing certain method +invocations (especially the very fast ones). This means that the invocation +count of methods is very innacurrate, but the total spent time (and CPU time) +should still provide some relavant metrics. + +### When to Use CPU Sampling +Unless you are interested in very precise performance metrics (albeit affected +by the added cost of instrumentation), you should use sampling most of the time. +The main advantage of profiling is its accuracy, but since there's the +performance impact added by instrumentation, most of the performance metrics +will be off by an unknown factor. + +### How to Run CPU Sampling + +### Local applications + +For local applications, launch VisualVM from the JDK binary directory, + +1. Select (double click) on the process that you would like to monitor on the + left hand screen. +2. Click on the "Sampler" tab +3. When you are ready to perform your test, select the button "CPU" next to the + "Sample" tab. +4. Once the test has finished, press "Stop" and press the "Snapshot" button. + +Please keep in mind that the data displayed before taking a snapshot may or may +not be very accurate. You should do your analysis only on snapshots. + +A common error that people do when following this steps is to take an actual +screenshot of the sampling screen. While it's nice they were so thoughtful, the +data is mostly pointless - as most of the time the performance bottlenecks will +be somewhere deeper in the call hierarchy and will not be seen from the +overview. + +### Remote applications + +For remote applications, the process is very similar but it requires setting up +a JMX connection to the Java process to be monitored. + +1. Enable the JMX port on your application. This is outside the scope of this + article, but you can check the official Oracle documentation for more + details: + [Monitoring and Management Using JMX Technology](https://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html) +2. Right click on the "Remote" tab in the left hand screen. +3. Select "Add Remote Host" +4. Fill in the host name. Most of the times, this will be sufficient but + depending on how you've enabled JMX on the remote process, you may need to + also check the "Advanced Settings" tab. +3. Right click on the newly added host and select "Add JMX connection". +4. Fill in the connection details (including port number) and the display name. + If you're application is deployed using multiple processes, you should enter + a descriptive display name. +5. Double click on the newly added JMX connection. +6. From this point onwards, the steps are identical with the ones from "Local + applications". + +### How to Interpet the Data +This depends a lot on the actual issues that you're trying to investigate and +the application architecture. For example, if you have a desktop application +with a fixed amount of threads, the call tree with the break down per thread may +be useful. + +However, if you're working on a web application with a variable number of +threads, it will probably be hard to figure out what's happening. In this case, +you should probably start from thet "Hot spots" tab and dig deeper from there. + +### Difference between "Self Time" and "Self Time (CPU)" +VisualVM reports two metrics related to the duration, but there is a significant +difference between them: + +- **self time** - counts the total time spent in that method, including the amount + of time spent on locks or other blocking behaviour +- **self time (cpu)** - counts the total time spent in that method, excluding the + amount of time the thread was blocked + +From here, you will need to decide on what you want to focus, + +- if you want to focus on optimising the multithreaded interactions, then you +should aim for the self time values including the time the threads were blocked +- if you're interested in the overall performace and not care too much about the +multithreaded interactions, then should focus solely on the self time (cpu). + +Be careful though on how you interpret your results. If you have a thread that +keeps a connection open, most likely you will see some very large numbers for +the self time. This is normal and it's not issue. + diff --git a/_posts/2018-01-07-wifi-without-networkmanager.md b/_posts/2018-01-07-wifi-without-networkmanager.md new file mode 100644 index 0000000..2d4765c --- /dev/null +++ b/_posts/2018-01-07-wifi-without-networkmanager.md @@ -0,0 +1,61 @@ +--- +layout: post +title: WiFi Networking without NetworkManager +date: 2018-01-07 +keywords: linux, networkmanager, wifi +--- + +The `NetworkManager.service` provides a simple way to configure WiFi networking on Linux systems, but it can cause issues when running in a custom setup (in my case with multiple network adapters). + +The following steps will disable the NetworkManager and allow configuring the WiFi through `wpa_supplicant`. They should work for most Linux distributions that use systemd (e.g. Debian, Raspbian, Kali Linux, etc.). + + + +### 1 Disable the NetworkManager service +```bash +systemctl disable NetworkManager +systemctl stop NetworkManager +``` + +### 2 WiFi configuration +Create `/etc/wpa_supplicant/wpa_supplicant.conf` based on the following template: + +``` +country=.. # ISO_3166 country code +update_config=1 +ctrl_interface=/var/run/wpa_supplicant + +network={ + scan_ssid=1 + ssid="Network SSID" # WiFi SSID to connect to + psk="password" # Password +} +``` + +### 3 Apply the WiFi configuration on boot +Add the following lines to `/etc/rc.local` or create the file if it does not exist. + +```bash +#!/bin/bash + +( + # Enable the network interface. Repeat this line for any other wifi interfaces + # that you want to use. + ip link set wlan0 up + + # Connect to the configured access point using the wlan0 interface + wpa_supplicant -B -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.conf + dhclient -v wlan0 + +) &> /var/log/local_inet.log & +``` + +If you've just created the `rc.local` file, you will also need to mark it as executable: +```bash +sudo chmod a+x /etc/rc.local +``` + +### 4 Reboot +```bash +sudo reboot +``` diff --git a/_sass/partials/_content.scss b/_sass/partials/_content.scss index c8b23ec..b80325f 100644 --- a/_sass/partials/_content.scss +++ b/_sass/partials/_content.scss @@ -27,3 +27,24 @@ article { .post-meta .date, .post-meta .category { padding-right: 1em; } + +.pagination { + li { + span { + color: $main-color; + // border-color: $main-color; + + #{$a-tags} { + color: $main-color; + } + } + } + .active { + span, span:focus, span:hover { + background-color: $main-color; + color: $highlight-color; + border-color: $main-color; + } + } + +} diff --git a/_sass/partials/_typography.scss b/_sass/partials/_typography.scss index b302268..08c6dc2 100644 --- a/_sass/partials/_typography.scss +++ b/_sass/partials/_typography.scss @@ -19,6 +19,27 @@ h1, h2, h3, h4, h5, h6 { h1 { padding-left: .5em; border-left: .5em solid $secondary-color; + font-size: 240%; +} + +h2 { + font-size: 200%; +} + +h3 { + font-size: 180%; +} + +h4 { + font-size: 160%; +} + +h5 { + font-size: 140%; +} + +h6 { + font-size: 120%; } blockquote { diff --git a/favicon.ico b/favicon.ico index 450cb02..a2d5097 100644 Binary files a/favicon.ico and b/favicon.ico differ diff --git a/img/greyfocus-logo-easter2-small.png b/img/greyfocus-logo-easter2-small.png new file mode 100644 index 0000000..7317286 Binary files /dev/null and b/img/greyfocus-logo-easter2-small.png differ diff --git a/img/greyfocus-logo-subtitle-small.png b/img/greyfocus-logo-subtitle-small.png deleted file mode 100644 index c08ace3..0000000 Binary files a/img/greyfocus-logo-subtitle-small.png and /dev/null differ