• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: Out of memory
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Out of memory


  • Subject: Re: Out of memory
  • From: Kieran Kelleher <email@hidden>
  • Date: Wed, 7 May 2008 12:56:50 -0400

Thanks Miguel .... I had not used jconsole before.... nice, simple and free.

I have some background processes that risk running out of memory such as importing millions of rows of data from CSV files and creating EO's. I know that EC recycling and many other strategies are recommended and for those processes I use such strategies, however, I have found that the use of the attached class saves my app from dying if it gets close to out of memory. All it does is check the memory usage and if too close for comfort it just forces garbage collection. Often this will push memory way down. This is probably of limited use where you have a big EOF operation inflating losts of EO's. If you memory problem is a gazillion sessions logged into the app, it may not help much at all.

It was inspired by some code I saw in Wonder at one stage. It was born out of desperation trying to get files with 10+ million rows to import and the usual strategies were not working and I really did not want to insert these as raw rows directly using adaptor operations since I wanted the full EOF validation features to skip non-conforming import rows.

Usage:
call the static method forceGarbageCollectionIfNecessary

Configuration using property. Less than 1.0 is a percentage memory usage threshold. Great than 1.0 is assumed a Megabyte free minimum threshold. Like I said look at Wonder code for a similar calculation somewhere. IIRC the difference between this and Wonder is that I put the run finalization inside the gc repeating loop.

YMMV,

Hope it is of use, Kieran



package wk.foundation;

import java.math.BigDecimal;

import org.apache.log4j.Logger;


public class WKRuntimeUtilities { private static final Logger log = Logger .getLogger(WKRuntimeUtilities.class);

/**
*@returntrueiffreememoryislessthanthethreshold
*/
public static boolean isMemoryStarved() {
String thresholdString = System.getProperty("wk.foundation.WKRuntimeUtilities.memoryThreshold", "0.9");
BigDecimal MEMORY_THRESHOLD = new BigDecimal(thresholdString);


long max = Runtime.getRuntime().maxMemory();
long total = Runtime.getRuntime().totalMemory();
long free = Runtime.getRuntime().freeMemory() + (max - total);
long used = max - free;
long threshold = (long) (MEMORY_THRESHOLD.doubleValue() < 1.0 ? MEMORY_THRESHOLD.doubleValue() * max : (max - (MEMORY_THRESHOLD.doubleValue() * 1024 * 1024)));



boolean result = (used > threshold);
if (log.isDebugEnabled())
log.debug("max = " + max + "; total = " + total + "; free = " + free + "; used = " + used + "; threshold = " + threshold
+ "; isMemoryStarved = " + result);
return result;
}


	public static String memoryStatistics() {
		long max = Runtime.getRuntime().maxMemory();
		long total = Runtime.getRuntime().totalMemory();
		long free = Runtime.getRuntime().freeMemory() + (max - total);
		long used = max - free;

StringBuilder s = new StringBuilder("Memory Statistics");
s.append("\nMax = " + WKFormatUtilities.userFriendlyByteCountString(max));
s.append("\nTotal = " + WKFormatUtilities.userFriendlyByteCountString(total));
s.append("\nFree = " + WKFormatUtilities.userFriendlyByteCountString(free));
s.append("\nUsed = " + WKFormatUtilities.userFriendlyByteCountString(used));
s.append("\nUsed Percent = " + used * 100 / max + " %");
return s.toString();
}


	/**
	 *Checksifmemoryisstarvedandthenforcesfullgarbagecollection
	 */
	public static void forceGarbageCollectionIfNecessary() {
		try {

if (WKRuntimeUtilities.isMemoryStarved()) {
StringBuilder s = new StringBuilder("Notification of Forced Garbage Collection");
s.append("\nBEFORE Forced GC = " + WKRuntimeUtilities.memoryStatistics());



WKRuntimeUtilities.forceGC(0);

s.append("\nAFTER Forced GC = " + WKRuntimeUtilities.memoryStatistics());

// For now log.error so I get emails, but when satisfied change to log.warn
log.error("NOT an ERROR - Just Notification so Kieran is aware of it.\n"
+ s.toString(), new Throwable());
}
} catch (Exception e) {
log.error("Error performing forced garbage collection. Check the code", e);
}


	}

    /**
     * Forces the garbage collector to run. The
     * max loop parameter determines the maximum
     * number of times to run the garbage collector
     * if the memory footprint is still going down.
     * In normal cases you would just need to call
     * this method with the parameter 1. If called
     * with the parameter 0 the garbage collector
     * will continue to run until no more free memory
     * is available to collect. <br/>
     * <br/>
     * Note: This can be a very costly operation and
     * should only be used in extreme circumstances.
     * @param maxLoop maximum times to run the garbage
     *		collector. Passing in 0 will cause the
     *		collector to run until all free objects
     *		have been collected.
     */
    public static void forceGC(int maxLoop) {
        log.warn("Forcing full Garbage Collection", new Throwable());
        Runtime runtime = Runtime.getRuntime();
        long isFree = runtime.freeMemory();
        long wasFree;
        int i=0;
        do {
            wasFree = isFree;
            runtime.gc();
            runtime.runFinalization();
            isFree = runtime.freeMemory();
            i++;
        } while (isFree > wasFree && (maxLoop<=0 || i<maxLoop) );
    }

}
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden


References: 
 >Re: Out of memory (From: David Griffith <email@hidden>)
 >Re: Out of memory (From: Miguel Arroz <email@hidden>)

  • Prev by Date: Re: WOLips Cheat Sheet
  • Next by Date: Re: WO Tutorials
  • Previous by thread: Re: Out of memory
  • Next by thread: Re: Out of memory
  • Index(es):
    • Date
    • Thread