【Dubbo】SPI加载配置流程

【Dubbo】SPI加载配置流程

Scroll Down

前言

请容许我先在这里调侃几句= =

早在了解微服务之处,我就稍微接触过一点Dubbo,只是那时候就只懂得怎么调API,对原理是一窍不通,源码就更没那实力去看了。

中途也学过Spring Cloud之类的框架,前段时间做课程设计也都用上了。那为啥我现在不去搞Spring Cloud而是来看Dubbo呢?

一方面主要是因为我所在的实习公司有关,现在公司的微服务框架核心就是RPC了,虽说是对Dubbo进行过一些改造,但原理还是大同小异的。另一方面,据说Dubbo通信有使用到Netty实现的,那就得趁Netty知识忘光之前康康了= =。

下面废话不多说,先从简单的开始学起,来看看Dubbo的SPI加载配置流程,这里再补充一下环境:

Dubbo Version:2.7.8 & 2.6.0

就加载配置这一块的源码和阿里2.6.X的版本差别不大,基本只是进一步封装了一下而已。

追踪源码

相信对Dubbo有一定了解的人,一定对下面这条语句很熟悉:

private static final Protocol PROTOCOL = ExtensionLoader.getExtensionLoader(Protocol.class).getAdaptiveExtension();

基本和SPI有关的东西,都离不开ExtensionLoader这个类,包括加载配置也一样,同时关于这里面包含了IOC和AOP的一些源码,这些坑等之后有机会再慢慢补充吧。

本文直接从如下代码位置开始看起:

org.apache.dubbo.common.extension.ExtensionLoader#getExtension(java.lang.String, boolean)
  • 如果你整合了SpringBoot,那么这个方法就是在refresh阶段被调用的,来到这方法之前大致就是:refresh->...->ExtensionLoader构造->反射->getExtension
  • 这中间的具体源码可以根据栈帧自行追踪,这里就不废话了。

来看看getExtension方法:
org.apache.dubbo.common.extension.ExtensionLoader#getExtension(java.lang.String, boolean)

    public T getExtension(String name, boolean wrap) {
        if (StringUtils.isEmpty(name)) {
            throw new IllegalArgumentException("Extension name == null");
        }
        if ("true".equals(name)) {
            return getDefaultExtension();
        }
        final Holder<Object> holder = getOrCreateHolder(name);
        Object instance = holder.get();
        if (instance == null) {
            synchronized (holder) {
                instance = holder.get();
                if (instance == null) {
                    // 跟进这里
                    instance = createExtension(name, wrap);
                    holder.set(instance);
                }
            }
        }
        return (T) instance;
    }

跟进createExtension方法:
org.apache.dubbo.common.extension.ExtensionLoader#createExtension

    @SuppressWarnings("unchecked")
    private T createExtension(String name, boolean wrap) {
        // 本节跟进getExtensionClasses
        Class<?> clazz = getExtensionClasses().get(name);
        if (clazz == null) {
            throw findException(name);
        }
        try {
            T instance = (T) EXTENSION_INSTANCES.get(clazz);
            if (instance == null) {
                EXTENSION_INSTANCES.putIfAbsent(clazz, clazz.newInstance());
                instance = (T) EXTENSION_INSTANCES.get(clazz);
            }
            // TODO 实现IOC核心,完成一些依赖注入的工作,里面顺便能看到ExtensionFactory,这块之后再补充
            injectExtension(instance);

            // TODO 下面代码貌似有AOP实现,同样之后再补充,这节不讲
            if (wrap) {

                List<Class<?>> wrapperClassesList = new ArrayList<>();
                if (cachedWrapperClasses != null) {
                    wrapperClassesList.addAll(cachedWrapperClasses);
                    wrapperClassesList.sort(WrapperComparator.COMPARATOR);
                    Collections.reverse(wrapperClassesList);
                }

                if (CollectionUtils.isNotEmpty(wrapperClassesList)) {
                    for (Class<?> wrapperClass : wrapperClassesList) {
                        Wrapper wrapper = wrapperClass.getAnnotation(Wrapper.class);
                        if (wrapper == null
                                || (ArrayUtils.contains(wrapper.matches(), name) && !ArrayUtils.contains(wrapper.mismatches(), name))) {
                            instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance));
                        }
                    }
                }
            }

            initExtension(instance);
            return instance;
        } catch (Throwable t) {
            throw new IllegalStateException("Extension instance (name: " + name + ", class: " +
                    type + ") couldn't be instantiated: " + t.getMessage(), t);
        }
    }

跟进getExtensionClasses方法
org.apache.dubbo.common.extension.ExtensionLoader#getExtensionClasses

    private Map<String, Class<?>> getExtensionClasses() {
        Map<String, Class<?>> classes = cachedClasses.get();
        if (classes == null) {
            synchronized (cachedClasses) {
                classes = cachedClasses.get();
                if (classes == null) {
                    // 跟进这里
                    classes = loadExtensionClasses();
                    cachedClasses.set(classes);
                }
            }
        }
        return classes;
    }
  • 稍微注意下这里出现的cachedClasses,因为下面还有机会遇到它。

继续跟进loadExtensionClasses方法:
org.apache.dubbo.common.extension.ExtensionLoader#loadExtensionClasses

    /**
     * synchronized in getExtensionClasses
     */
    private Map<String, Class<?>> loadExtensionClasses() {
        // 从SPI上获取名字缓存
        cacheDefaultExtensionName();

        Map<String, Class<?>> extensionClasses = new HashMap<>();

        for (LoadingStrategy strategy : strategies) {
            loadDirectory(extensionClasses, strategy.directory(), type.getName(), strategy.preferExtensionClassLoader(), strategy.overridden(), strategy.excludedPackages());
            loadDirectory(extensionClasses, strategy.directory(), type.getName().replace("org.apache", "com.alibaba"), strategy.preferExtensionClassLoader(), strategy.overridden(), strategy.excludedPackages());
        }

        // 返回之后会被缓存进cachedClasses
        return extensionClasses;
    }

注意,上面这段代码在2.6.0的版本中是长这样子,虽然皮换了下,但是大致原理还是相同的:

    // synchronized in getExtensionClasses
    private Map<String, Class<?>> loadExtensionClasses() {
        final SPI defaultAnnotation = type.getAnnotation(SPI.class);
        if (defaultAnnotation != null) {
            String value = defaultAnnotation.value();
            if (value != null && (value = value.trim()).length() > 0) {
                String[] names = NAME_SEPARATOR.split(value);
                if (names.length > 1) {
                    throw new IllegalStateException("more than 1 default extension name on extension " + type.getName()
                            + ": " + Arrays.toString(names));
                }
                if (names.length == 1) cachedDefaultName = names[0];
            }
        }

        Map<String, Class<?>> extensionClasses = new HashMap<String, Class<?>>();
        loadFile(extensionClasses, DUBBO_INTERNAL_DIRECTORY);
        loadFile(extensionClasses, DUBBO_DIRECTORY);
        loadFile(extensionClasses, SERVICES_DIRECTORY);
        return extensionClasses;
    }
  • 一部分逻辑在2.7.8版本中被封装到了cacheDefaultExtensionName方法中了,另外核心的loadFile也变成了loadDirectory。

继续跟进2.7.8版本的loadDirectory方法:
org.apache.dubbo.common.extension.ExtensionLoader#loadDirectory(java.util.Map<java.lang.String,java.lang.Class<?>>, java.lang.String, java.lang.String, boolean, boolean, java.lang.String...)

    private void loadDirectory(Map<String, Class<?>> extensionClasses, String dir, String type,
                               boolean extensionLoaderClassLoaderFirst, boolean overridden, String... excludedPackages) {
        // type为全面获得的类的全路径名
        String fileName = dir + type;
        try {
            Enumeration<java.net.URL> urls = null;
            ClassLoader classLoader = findClassLoader();

            // try to load from ExtensionLoader's ClassLoader first
            if (extensionLoaderClassLoaderFirst) {
                ClassLoader extensionLoaderClassLoader = ExtensionLoader.class.getClassLoader();
                if (ClassLoader.getSystemClassLoader() != extensionLoaderClassLoader) {
                    urls = extensionLoaderClassLoader.getResources(fileName);
                }
            }

            if (urls == null || !urls.hasMoreElements()) {
                if (classLoader != null) {
                    urls = classLoader.getResources(fileName);
                } else {
                    urls = ClassLoader.getSystemResources(fileName);
                }
            }

            if (urls != null) {
                while (urls.hasMoreElements()) {
                    java.net.URL resourceURL = urls.nextElement();
                    // 跟进这里,里核心不远了
                    loadResource(extensionClasses, classLoader, resourceURL, overridden, excludedPackages);
                }
            }
        } catch (Throwable t) {
            logger.error("Exception occurred when loading extension class (interface: " +
                    type + ", description file: " + fileName + ").", t);
        }
    }

上面的fileName其实就是SPI描述文件,SPI的描述文件一般都在META-INF/dubbo/internal/orMETA-INF/services/等目录下(如果你对SpringBoot自动装配有过了解,相信这种模式你并不会感到陌生),比如我们熟悉的DubboProtocol:

DubboProtocol.png

又或是RegistryProtocol,你都可以在这里面找到:

RegistryProtocol.png

比如拿2.6.0版本打个断点就能看到fileName是这样子的:
fileName.png

之后我们继续跟进loadResource方法:
org.apache.dubbo.common.extension.ExtensionLoader#loadResource

    private void loadResource(Map<String, Class<?>> extensionClasses, ClassLoader classLoader,
                              java.net.URL resourceURL, boolean overridden, String... excludedPackages) {
        try {
            // 读取SPI描述文件,并解析
            try (BufferedReader reader = new BufferedReader(new InputStreamReader(resourceURL.openStream(), StandardCharsets.UTF_8))) {
                String line;
                while ((line = reader.readLine()) != null) {
                    final int ci = line.indexOf('#');
                    if (ci >= 0) {
                        line = line.substring(0, ci);
                    }
                    line = line.trim();
                    if (line.length() > 0) {
                        try {
                            String name = null;
                            int i = line.indexOf('=');
                            if (i > 0) {
                                name = line.substring(0, i).trim();
                                line = line.substring(i + 1).trim();
                            }
                            if (line.length() > 0 && !isExcluded(line, excludedPackages)) {
                                // Class.forName(line, true, classLoader) 在JDK中加载每一行读到的类
                                // 继续跟进这个方法
                                loadClass(extensionClasses, resourceURL, Class.forName(line, true, classLoader), name, overridden);
                            }
                        } catch (Throwable t) {
                            IllegalStateException e = new IllegalStateException("Failed to load extension class (interface: " + type + ", class line: " + line + ") in " + resourceURL + ", cause: " + t.getMessage(), t);
                            exceptions.put(line, e);
                        }
                    }
                }
            }
        } catch (Throwable t) {
            logger.error("Exception occurred when loading extension class (interface: " +
                    type + ", class file: " + resourceURL + ") in " + resourceURL, t);
        }
    }

继续跟进loadClass,基本就是最后了:
org.apache.dubbo.common.extension.ExtensionLoader#loadClass

    private void loadClass(Map<String, Class<?>> extensionClasses, java.net.URL resourceURL, Class<?> clazz, String name,
                           boolean overridden) throws NoSuchMethodException {
        // 确保读到的类都是SPI的实现类,否则抛出异常
        if (!type.isAssignableFrom(clazz)) {
            throw new IllegalStateException("Error occurred when loading extension class (interface: " +
                    type + ", class line: " + clazz.getName() + "), class "
                    + clazz.getName() + " is not subtype of interface.");
        }
        // 如果实现类上有Adaptive注解,则将读到的类赋值给 ExtensionLoader 类的实例属性 cacheAdaptiveClass
        if (clazz.isAnnotationPresent(Adaptive.class)) {
            cacheAdaptiveClass(clazz, overridden);
        } else if (isWrapperClass(clazz)) {
            // 如果实现类上没有Adaptive注解,但有带参数的构造函数存在,则会先放到 cacheWrapperClass 中
            cacheWrapperClass(clazz);
        } else {
            // 如果只有空参构造函数
            clazz.getConstructor();
            if (StringUtils.isEmpty(name)) {
                name = findAnnotationName(clazz);
                if (name.length() == 0) {
                    throw new IllegalStateException("No such extension name for the class " + clazz.getName() + " in the config " + resourceURL);
                }
            }

            String[] names = NAME_SEPARATOR.split(name);
            if (ArrayUtils.isNotEmpty(names)) {
                // 判断类上有没有Activate注解,如果有则加入到 cachedActivates 缓存中
                cacheActivateClass(clazz, names[0]);
                for (String n : names) {
                    // clazz 为 key , name 为 value 进行缓存
                    cacheName(clazz, n);
                    // name 为 key ,clazz 为 value 满足条件后缓存(算是间接影响cachedClasses)
                    saveInExtensionClass(extensionClasses, clazz, n, overridden);
                }
            }
        }
    }
  • 基本就是判断实现类的注解、构造函数,放入不同的缓存中。

值得注意的是saveInExtensionClass这个方法,它就是将剩余符合条件的clazz放到extensionClasses中,而extensionClasses最终是被填充到上面getExtensionClasses遇到过的cachedClasses中。

到此,SPI加载描述文件的实现类基本就到此结束了,我不排除还有些遗漏的,如果有的话之后再补上吧~

小结

加载流程其实看着也比较简单,核心其实就是loadClass方法,它主要做了以下事情

  • 带Adaptive注解的实现会被缓存到cachedAdaptiveClass
  • 没有Adaptive注解的,但有有参构造函数的,会放到cachedWrapperClass
  • 没有Adaptive注解的,也没有有参构造函数的,会放到cachedActivates
  • 剩下的会缓存cachedNames中,满足特定条件则在缓存到cachedClasses中。

至于这些缓存都是什么时候用到,那就是后面源码的故事了。