PermGen Vs Metaspace

As you saw in Java 8 PermGen space has been decommission. In JDK 8, classes metadata is now stored in the native heap and this space is called Metaspace.

I recommend that you read the PermGen removal summary and comments from Jon on this subject. 

In summary:

PermGen space situation

• This memory space is completely removed.
• The PermSize and MaxPermSize JVM arguments are ignored and a warning is issued if present at start-up.

Metaspace memory allocation model

• Most allocations for the class metadata are now allocated out of native memory.
• The klasses that were used to describe class metadata have been removed.

Metaspace capacity

• By default class metadata allocation is limited by the amount of available native memory (capacity will of course depend if you use a 32-bit JVM vs. 64-bit along with OS virtual memory availability).
• A new flag is available (MaxMetaspaceSize), allowing you to limit the amount of native memory used for class metadata. If you don’t specify this flag, the Metaspace will dynamically re-size depending of the application demand at runtime.

Metaspace garbage collection

• Garbage collection of the dead classes and classloaders is triggered once the class metadata usage reaches the “MaxMetaspaceSize”.
• Proper monitoring & tuning of the Metaspace will obviously be required in order to limit the frequency or delay of such garbage collections. Excessive Metaspace garbage collections may be a symptom of classes, classloaders memory leak or inadequate sizing for your application.

Java heap space impact

• Some miscellaneous data has been moved to the Java heap space. This means you may observe an increase of the Java heap space following a future JDK 8 upgrade.

Metaspace monitoring

• Metaspace usage is available from the HotSpot 1.8 verbose GC log output.
• Jstat & JVisualVM have not been updated at this point based on our testing with b75 and the old PermGen space references are still present.

How to set the Metaspace in Java 8

There are some new flags added for Metaspace in JDK 8:

• -XX:MetaspaceSize=

where is the initial amount of space(the initial high-water-mark) allocated for class metadata (in bytes) that may induce a garbage collection to unload classes. The amount is approximate. After the high-water-mark is first reached, the next high-water-mark is managed by the garbage collector

• -XX:MaxMetaspaceSize=

where is the maximum amount of space to be allocated for class metadata (in bytes). This flag can be used to limit the amount of space allocated for class metadata. This value is approximate. By default there is no limit set.

• -XX:MinMetaspaceFreeRatio=

where is the minimum percentage of class metadata capacity free after a GC to avoid an increase in the amount of space (high-water-mark) allocated for class metadata that will induce a garbage collection.

• -XX:MaxMetaspaceFreeRatio=

where is the maximum percentage of class metadata capacity free after a GC to avoid a reduction in the amount of space (high-water-mark) allocated for class metadata that will induce a garbage collection.

permgen vs metaspace in java 8

As you may be aware in Java 8, one of its feature is removal of Permanent Generation (PermGen).
In this post, we will see how and for what the PermGen was being used till java 7 and its successor in java 8

A Java memory problem such as java.lang.OutOfMemoryError: PermGen space is one of the most frequent and complex problems a Java EE application support person can face with a production system. This article is one of the post that will focus on a particular OOM flavour: PermGen space depletion of a Java HotSpot VM.

Parameter used: -XXPermSize=1024m

before we see more details about it lets see a memory model in java:

 

Primary objective is to revisit the fundamentals of the permanent generation space and to teach you how to identify a particular pattern of PermGen space problem and possible causes of PermGen memory leak.

 

java.lang.OutOfMemoryError: PermGen space patterns

Find below some of the most common patterns of OutOfMemoryError due to the depletion of the PermGen space.

Pattern	Symptoms	Possible root cause scenarios	Resolution
OOM observed during or after a migration of a Java EE server to newer version	- OOM may be observed on server start-up at deployment time - OOM may be observed very shortly after server start-up and after 1 or 2+ hours of production traffic	- Higher PermGen capacity is often required due to increased Java EE server vendor code and libraries	- Increase your PermGen space capacity via -XX:MaxPermSize
OOM observed after a certain period of time	- OOM observed after a longer but consistent period of time (days) - PermGen space monitoring will show hourly or daily increase during your application business hours	- There are many possible causes of PermGen space memory leak. The most common is a class loader leak: increasing number of Class objects overtime - Improper JVM arguments like usage of the Xnoclassgc flag (turn OFF Class garbage collection)	- Review your JVM HotSpot start-up arguments for any obvious problem like Xnoclassgc flag - Analyse the JVM HotSpot Heap Dump as it can provides some hints on the source of a class loader leak - Investigate any third party API you are using for any potential class loader leak defect - Investigate your application code for any improper use of Reflection API and / or dynamic class loading
OOM observed following a redeploy of your application code (EAR, WAR files...)	- OOM may be observed during or shortly after your application redeploy process	- Unloading and reloading of your application code can lead to PermGen leak (class loader leak) and deplete your PermGen space fairly quickly	- Open a ticket with your Java EE vendor for any known class loader leak issue - Shutdown and restart your server (JVM) post deployment to cleanup any class loader leak

  

 

 

In next part we will see more details about metaspace introduced in java 8 as a successor of PermGen.

Logging levels in Log4j

As promised in previous blog, I have collected some information about log levels and its configurations.

There are mainly eight log level's available in log4j.

ALL, TRACE, DEBUG , INFO, WARN, ERROR, FATAL, OFF

  TRACE is used for finer-grained informational events than the DEBUG.

We can define more than log levels in our applications but one per appender. Following is a simple example of how it look like:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

import org.apache.log4j.*; public class LogClass { private static org.apache.log4j.Logger log = Logger.getLogger(LogClass.class); public static void main(String[] args) { log.setLevel(Level.WARN); log.trace("Trace Message!"); log.debug("Debug Message!"); log.info("Info Message!"); log.warn("Warn Message!"); log.error("Error Message!"); log.fatal("Fatal Message!"); } }

# Define the root logger with appender file
log = /usr/home/log4j
log4j.rootLogger = WARN, FILE

# Define the file appender
log4j.appender.FILE=org.apache.log4j.FileAppender
log4j.appender.FILE.File=${log}/log.out

# Define the layout for file appender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern=%m%n

Now if we compile and run the above program and we will get following result in /usr/home/log4j/log.out file.

Warn Message!
Error Message!
Fatal Message!

 If someone needs more information do add a comment, I will try to provide that information as well.


Happy reading and keep adding comments. cheers..!

Logging with Mapped Diagnostic Context(MDC)

I was trying to write something on this topic from very long time since this is less known but very useful technology in case of high performance, multi threaded distributed systems.

What is MDC or NDC or ThreadContext(log4j 2)?

We all know that log4j is a widely accepted and most powerful logging mechanism in today's world. No application can be written without log4j. But its always messy since it works on the principle of log levels i.e. ALL< DEBUG < INFO < WARN < ERROR < FATAL < OFF. We will see how log level works in a different  post, Here

It generates so many messages in the log files depending on the log level that finding the real cause becomes very tedious.

MDC is a concept or feature of Log4j logging library which can be used to group related log messages together

Log4j also provides a similar utility called as NDC, known as Nested Diagnostic Context

Both of the above can be replaced with ThreadContext class in log4j 2.

The ThreadContext class provides a Map and a Set to replace MDC and NDC. The Thread Context Map allows any number of items to be added and be identified using key/value pairs, while ThreadContextStack allows one or more items to be pushed on the Stack and then be identified by their order in the Stack or by the data itself. 

Remember MDC is managed on a per thread basis and every child thread automatically inherits a copy of mapped diagonstice context from its parent. This is achieved by InheritableThreadLocal  i which is a sub class of  ThreadLocal class.

How to use MDC to log information in log4j

 You can put any information into Mapped Diagnostic Context or MDC by calling the put() method. MDC is a static class i.e. a class with just static methods, which means you can directly call them without creating any instance of MDC.

Remember, this information is stored as thread local variable, which may cause a memory leak in a web application if used incorrectly (see). If you are using MDC to log client or order specific information e.g. orderId in a web application using a filter than you can also remove it once done.

try{
  MDC.put("tradeId", trade.getId());
}finally{
  MDC.remove.remove("tradeId");
}

By the way from log4j2 onwards, MDC and NDC (Nested Diagnostic Context) are merged into a class called ThreadContext So, instead of using MDC, you need to use ThreadContext class to put unique context data.

try{
   ThreadContext.put("tradeId", trade.getId());
}finally {
   ThreadContext.clear();
}

Once available in MDC, this information can be printed in the log file using PatternLayout. You can use %X{tradeId) to print tradeId in log messages, remember tradeId for different trades will be different, which allows you to trace logging messages for individual trades. 

This is same as we print the logs in log4j. We will see pattern layout in details here in details.


I will share a complete example in my coming blogs. Stay connected and happy reading :)

From LocalDate to LocalDateTime

How to convert LocalDate to LocalDateTime?

Local Date and Local DateTime is introduced in Java 8 under package java.time.

First we need to understand what we have and what we want to get.

From LocalDate we have: Instant + System default time zone.
What we want to have is: Instant + System default time zone + Local Time

LocalDateTime.of() methods gives us so many options to create LocalDateTime. On of the option is as given below

LocalDate localDate = LocalDate.now()

LocalDateTime ldt= LocalDateTime.of(localDate, LocalTime.MIN);

Here LocalTime.Min will give us the Day's start time i.e. 00:00:00
To get LocalDateTime with current time we can use following:

LocalDateTime localDateTimeNow = LocalDateTime.of(localDate, LocalTime.now());

Example:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

import java.time.Instant; import java.time.LocalDate; import java.time.LocalDateTime; import java.time.LocalTime; import java.time.ZoneId; import java.util.Date; public class LocalDateToLocalDateTime { public static void main(String[] args) { // Asia/Kuala_Lumpur +8 ZoneId defaultZoneId = ZoneId.systemDefault(); System.out.println("System Default TimeZone : " + defaultZoneId); Date date = new Date(); System.out.println("date : " + date); // 1. Convert Date -> Instant Instant instant = date.toInstant(); System.out.println("instant : " + instant); // Zone : UTC+0 // 2. Instant + system default time zone + toLocalDate() = LocalDate LocalDate localDate = instant.atZone(defaultZoneId).toLocalDate(); System.out.println("localDate : " + localDate); // 3. Instant + system default time zone + toLocalDateTime() = // LocalDateTime LocalDateTime localDateTime = instant.atZone(defaultZoneId).toLocalDateTime(); System.out.println("localDateTime : " + localDateTime); LocalDateTime ldt = LocalDateTime.of(localDate, LocalTime.now()); System.out.println("localDateTime : " + ldt); } }

Using Lambda Expressions to sort a collection in Java 8

Below is an example to show how to use Lambda expression to sort a collection.

Prior to Java 8, it was necessary to implement the java.util.Comparator interface with an anonymous (or named) class when sorting a collection:

Java SE 1.2

Collections.sort(
    personList,
    new Comparator<Person>() {
        public int compare(Person p1, Person p2){
            return p1.getFirstName().compareTo(p2.getFirstName());
        }
    }
);

Starting with Java 8, the anonymous class can be replaced with a lambda expression. Note that the types for the parameters p1 and p2 can be left out, as the compiler will infer them automatically:

Collections.sort(
    personList, 
    (p1, p2) -> p1.getFirstName().compareTo(p2.getFirstName())
);

The example can be simplified by using Comparator.comparing and method references expressed using the :: (double colon) symbol.

Collections.sort(
    personList,
    Comparator.comparing(Person::getFirstName)
);

A static import allows us to express this more concisely, but it is debatable whether this improves overall readability:

import static java.util.Collections.sort;
import static java.util.Comparator.comparing;
//...
sort(personList, comparing(Person::getFirstName));

Comparators built this way can also be chained together. For example, after comparing people by their first name, if there are people with the same first name, the thenComparing method with also compare by last name:

sort(personList, comparing(Person::getFirstName).thenComparing(Person::getLastName));

Scheduling a Job using spring

Requirement:

We have to send email on a regular interval by checking a condition automatically.

Solution:
We can use Queartz Job scheduler which is an open source job scheduling library and can be attached with any java application.

Alternatively we can use Spring's Task scheduler using TaskScheduler and TaskExecutor.

Pros:

• This approach is good for the situation where you are already using Spring in code. 
• Its quick and easy to implement.
• You do not have many jobs to run.

To implement the spring way we should use @EnableScheduling annotation in our AppConfig.java
The above annotation enables Spring's scheduled task execution capability.

Below is an example which gets executed every hours and prints the current time.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

package blogspot.thinkwithjava; import java.text.SimpleDateFormat; import java.util.Date; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.scheduling.annotation.Scheduled; import org.springframework.stereotype.Component; /** * * @author R.D * */ @Component public class ScheduledTasks { private static final Logger log = LoggerFactory.getLogger(ScheduledTasks.class); private static final SimpleDateFormat dateFormat = new SimpleDateFormat("HH:mm:ss"); @Scheduled(cron = "0 0 0/1 1/1 * ?") public void reportCurrentTime() { log.info("The time is now {}", dateFormat.format(new Date())); } }

cron = "0 0 0/1 1/1 * ?"

To schedule a task we provide a cron pattern as mentioned above. The pattern is a list of six single space-separated fields: representing second, minute, hour, day, month, weekday. Month and weekday names can be given as the first three letters of the English names.

Cron pattern for every Five Minute

"0 0/5 * 1/1 * ?"

Cron pattern for every hour

 "0 0 0/1 1/1 * ?"

Cron pattern for every four hour

"0 0 0/4 1/1 * ?"

More information is given here.