https://www.ibm.com/developerworks/cn/java/l-multithreading/
接自定义类加载器的理论,讲一个实践。
我们都有使用jsp的经验,为什么jsp可以修改后直接生效?就是ClassLoader在起作用,一个jsp对应一个ClassLoader,一旦jsp修改,就需要卸载原来加载此jsp(先是被转换为java文件,然后被编译为class文件)的ClassLoader ,然后重新生成一个ClassLoader来加载jsp对应的class文件。
最近参与到了一个抓取垂直网站报价数据的设计,由于抓取的网站有上千之多,并且每天都有大量的网站更新,导致原来的java解析代码必须修改以适应原来的功能。
数据抓取有两步:
1、抓取数据页面(html,或者json串)
2、解析数据
这样我们的系统,对每一要抓取的个网站建一个类,根据条件调用不同的类对象,问题来了:如果修改其中一个网站的类,如何生效? 重新启动tomcat当然是可以的,不过代价过高,也不可取。
想必大家想到了jsp和tomcat交互的方式:通过对每一个类建立一个ClassLoader对象,如果某个类更新了,上传class文件到特定目录下,重新加载一个ClassLoader对象,由新的ClassLoader来加载class,然后生成实例,处理请求。
下面我附上相关核心代码:
- public abstract class CachedClassLoader extends ClassLoader {
- private final static Log logger = LogFactory.getLog(CachedClassLoader.class);
- protected HashMap<String,Class<?>> cache = null;
- protected String classname;
- protected String path;
- public String getPath() {
- return path;
- }
- public CachedClassLoader(String path, String classname) {
- super();
- this.classname = classname;
- this.path = path;
- this.cache = new HashMap<String,Class<?>>();
- }
- /**
- * Loads the class with the specified name.
- * @param name: classname.
- */
- public synchronized Class<?> loadClass(String classname, boolean resolve) {
- if (this.cache.containsKey(classname)) {
- logger.debug("load Class:" + classname + " from cache.");
- Class<?> c = this.cache.get(classname);
- if (resolve)
- resolveClass(c);
- return c;
- } else {
- try {
- Class<?> c = Class.forName(classname);
- return c;
- }
- catch (ClassNotFoundException e) {
- Class<?> c = this.newClass(classname);
- if (c == null)
- return null;
- this.cache.put(classname, c);
- if (resolve)
- resolveClass(c);
- return c;
- }
- catch (NoClassDefFoundError e) {
- Class<?> c = this.newClass(classname);
- if (c == null)
- return null;
- this.cache.put(classname, c);
- if (resolve)
- resolveClass(c);
- return c;
- }
- }
- }
- public synchronized Class<?> getClass(String classname){
- return this.cache.get(classname);
- }
- /**
- * @return java.lang.Class
- * @param name
- * @param resolve
- */
- public synchronized Class<?> loadClass(boolean resolve) {
- return this.loadClass(this.classname, resolve);
- }
- /**
- * Abstract method for create new class object.
- * @param classname
- * @return
- */
- abstract Class<?> newClass(String classname);
- public String getClassname() {
- return classname;
- }
- public void setClassname(String classname) {
- this.classname = classname;
- }
- }
- public class FileClassLoader extends CachedClassLoader{
- private static Log logger =LogFactory.getLog(FileClassLoader.class);
- public String CLASSPATH_ROOT=TClassLoaderFactory.getFactory().getPropertyValue(TClassLoaderFactory.FILEROOT_PATH);
- public FileClassLoader (String path,String classname) {
- super(path, classname);
- }
- /**
- * Implements CachedClassLoader.newClass method.
- * @param classname
- */
- protected Class<?> newClass(String classname) {
- String fullpath = CLASSPATH_ROOT+File.separator+this.path+File.separator+classname + ".class";
- logger.debug("loading remote class " + classname + " from "+ fullpath);
- byte data[] = loadClassData(fullpath);
- if (data == null) {
- logger.debug("Class data is null");
- return null;
- }
- logger.debug("defining class " + classname);
- try {
- return super.defineClass(this.path.replaceAll("\\\\", ".")+"."+classname, data, 0, data.length);
- } catch (Exception e) {
- logger.error("Init class exception",e);
- }
- return null;
- }
- /**
- * Read class as a byts array.
- * @return byte[]
- * @param name
- */
- private byte[] loadClassData(String urlString) {
- logger.debug("loadClassData by:"+urlString);
- try {
- //return byteOutput.toByteArray();
- FileInputStream in =new FileInputStream(urlString);
- ByteArrayOutputStream out = new ByteArrayOutputStream();
- FileChannel channel =in.getChannel();
- WritableByteChannel outchannel = Channels.newChannel(out);
- ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
- while (true) {
- int i = channel.read(buffer);
- if (i == 0 || i == -1) {
- break;
- }
- buffer.flip();
- outchannel.write(buffer);
- buffer.clear();
- }
- byte[] bytes =out.toByteArray();
- out.close();
- in.close();
- return bytes;
- } catch (IOException ie) {
- logger.error("read local file exception "+urlString, ie);
- }
- return null;
- }
- /**
- * Load spec file from FileClassLoader's rootpath.
- * @param name resource's name.
- */
- public InputStream getResourceAsStream(String name) {
- String fullpath = CLASSPATH_ROOT+File.separator+this.path+File.separator+name;
- logger.debug("load resource from:"+fullpath);
- try {
- return new FileInputStream(fullpath);
- }
- catch(FileNotFoundException fe) {
- logger.error("spec:"+fullpath,fe);
- return null;
- }
- }
- }
- public class ClassLoaderFactory {
- private static Log logger =LogFactory.getLog(ClassLoaderFactory.class);
- public static final String LOADER_NAME = "classloader.name";
- public static final String NETROOT_URL = "classloader.NetworkClassLoader";
- public static final String FILEROOT_PATH = "classloader.FileClassLoader";
- public static final String PREVIOUS_ON_FILE="classloader.FileClassLoader.PRELOAD";
- private Map<String,ClassLoader> loaderMap = null;
- private static ClassLoaderFactory factory = new ClassLoaderFactory();
- public Properties conf = new Properties();
- private ClassLoaderFactory(){
- this.loaderMap = new ConcurrentHashMap<String,ClassLoader>();
- InputStream in = FileClassLoader.class.getResourceAsStream("/*****.properties");
- try {
- conf.load(in);
- logger.debug("ClassLoaderFactory init:"+LOADER_NAME +this.conf.getProperty(LOADER_NAME) );
- logger.debug("ClassLoaderFactory init:"+NETROOT_URL + this.conf.getProperty(NETROOT_URL) );
- logger.debug("ClassLoaderFactory init:"+FILEROOT_PATH + this.conf.getProperty(FILEROOT_PATH));
- }
- catch(IOException ie) {
- logger.error("Init classpath exception",ie);
- }
- }
- /**
- * Implements factory pattern.
- * @return
- */
- public static ClassLoaderFactory getFactory() {
- return factory;
- }
- protected String getPropertyValue(String propertyName) {
- return this.conf.getProperty(propertyName);
- }
- /**
- * Create new classloader object for this wrapper. default classloader is FileClassLoader.
- * @param key
- * @param classname
- * @return
- */
- public ClassLoader getClassLoader(String key,String classname) {
- long startTime = System.currentTimeMillis();
- String loaderKey = key;
- if(!this.loaderMap.containsKey(loaderKey)){
- synchronized(this.loaderMap) {
- if(this.loaderMap.containsKey(loaderKey)){
- return (ClassLoader)this.loaderMap.get(loaderKey);
- }
- try {
- Class<?> cl = Class.forName(this.conf.getProperty(LOADER_NAME));
- Class<?>[] params = {String.class,String.class};
- Constructor<?> constructor = cl.getConstructor(params);
- String[] args = {key,classname};
- logger.info("create new ClassLoader for:"+key+" classname:"+classname+" consume:"+(System.currentTimeMillis()-startTime)+" (ms)");
- this.loaderMap.put(loaderKey, (ClassLoader)constructor.newInstance(args));
- }
- catch(ClassNotFoundException cne) {
- logger.error("init classloader failed. system occure fetal error.!!!"+key+" codename:"+classname, cne);
- }
- catch(NoSuchMethodException nme) {
- logger.error("key:"+key+" classname:"+classname+ "get classloader failed.",nme);
- }
- catch(Exception e){
- logger.error("key:"+key+" classname:"+classname+ "get classloader failed.",e);
- }
- }
- }else {
- //(ClassLoader)this.loaderMap.get(loaderKey);
- CachedClassLoader loader =(CachedClassLoader)this.loaderMap.get(loaderKey);
- loader.setClassname(classname);
- logger.debug("retrieve classloader from cache map, key:"+key+" classname:"+classname);
- }
- return (ClassLoader)this.loaderMap.get(loaderKey);
- }
- public void reload(String key){
- if(loaderMap.containsKey(key)){
- synchronized(this.loaderMap) {
- loaderMap.remove(key);
- logger.info("Wrapper classes for key:"+key+ " were removed!");
- }
- }
- }
- public void reloadAll() {
- synchronized (this.loaderMap) {
- loaderMap.clear();
- logger.info("Wrapper classes for all key were removed!");
- }
- }
- }
- /**
- *
- * @author xinchun.wang
- @email: 532002108@qq.com
- * @createTime 2015-4-4 下午9:54:12
- */
- @Controller
- @RequestMapping("test")
- public class TestController {
- private final Logger logger = LoggerFactory.getLogger(getClass());
- private ClassLoaderFactory classLoaderFactory = TClassLoaderFactory.getFactory();
- private static final String path = "com"+File.separator+"gym"+File.separator+"backadmin"+File.separator+"service"+File.separator+"user";
- @SuppressWarnings("unchecked")
- @RequestMapping("getData")
- @ResponseBody
- public Map<String, Object> getData() throws Exception {
- logger.info("enter getData");
- CachedClassLoader loader = (CachedClassLoader)classLoaderFactory.getClassLoader(path, "BasicUserService");
- Class<UserService> userServiceClass = (Class<UserService>)loader.getClass("BasicUserService");
- UserService userService = userServiceClass.newInstance();
- System.out.println(userService.getClass().getClassLoader());
- Map<String, Object> model = userService.getUser();
- logger.info("exit getData");
- return model;
- }
- @RequestMapping("reload")
- @ResponseBody
- public Map<String, Object> reload(String classname) throws Exception {
- Map<String, Object> model = new HashMap<String,Object>();
- try{
- classLoaderFactory.reload(path);
- CachedClassLoader loader = (CachedClassLoader)classLoaderFactory.getClassLoader(path, "BasicUserService");
- loader.loadClass(classname);
- model.put("ret", "success");
- }catch(Exception e){
- logger.error("",e);
- model.put("ret", e);
- }
- return model;
- }
- }
- /**
- *
- * @author xinchun.wang
- @email: 532002108@qq.com
- */
- public class BasicUserService implements UserService {
- public Map<String, Object> getUser() {
- Map<String, Object> model = new HashMap<String, Object>();
- model.put("username", "ooooooooooooo");
- return model;
- }
- }
测试:随意更改BasicUserService 的实现,然后调用reload。
相关推荐
classloader教程 --- from IBM
LazyWorker.zip,一个智能控制进入电子邮件地址,自动检查是否存在域,如果域可以接收电子邮件。LaZyWork是一个帮助类延迟任务。例如,检查需要网络操作且不应在每次按键关闭后进行的输入。
>se.jiderhamn.classloader-leak-prevention</ groupId > < artifactId >classloader-leak-prevention-servlet3</ artifactId > < version >2.7.0</ version > </ dependency > 如果您遇到 ...
轻量级ndk实用程序,可帮助绕过Android N的classloader-namespace限制ndk_dlopen轻量级ndk实用程序,可帮助绕过Android N的classloader-namespace限制技术OSR(堆栈替换)支持x86,x86_64,armeabi-v7a(拇指和手臂...
classloader-playground, 一个简单的java依赖隔离容器类
【IT十八掌徐培成】Java基础第25天-04.classLoader-系统资源-不可见类访问.zip
jar包,官方版本,自测可用
jar包,官方版本,自测可用
jar包,官方版本,自测可用
jar包,官方版本,自测可用
jar包,官方版本,自测可用
jar包,官方版本,自测可用
支持动态的重新加载一个已经存在的类,并且讲之前的旧的类卸载掉。
TubeSock.zip,用java a实现的websocket客户端用java实现的websocket客户端库
jar包,官方版本,自测可用
关于J2EE服务器的ClassLoader的原理,该文档清晰了揭示了jvm装载类的顺序,同时用户可以自定义修改classLoader的配置 通过该文档,可以加深对Java虚拟机的理解
Java中ClassLoader的解析,从ClassLoader的角度分析了JVM,装载类,创建类的对象的整个过程,更清晰的了解JVM的运行机制。
gs-classloader-inspector 使用 JMX 浏览千兆空间类加载器 启用将 -javaagent:ClassloaderAgent.jar 添加到 JVM 参数识别 Gigaspaces 类加载器并显示附加信息
java classloader 讲义-淘宝网
Java 自定义ClassLoader 实现类的热替换核心代码