Skip to content

Commit 9ec86f8

Browse files
authored
test2 (#14)
* combine test coverage report (apache#2643) * Optimize ut for serialization model. (apache#2632) * 1. Integrate the unit test of the serialization module, code reuse. 2. Remove redundant unit tests and pojo classes. 3. Remove the useless pojo class in the common module. * fix ci fail * fix ci fail * fix ci fail * fix ci fail * remove author info * prototype for issue2570 (apache#2640) * apache#2570: dubbo all in one fail to start from a tomcat server when spring framework is absent * add comments and fix unit test * add license header * update comments in unit tests * [Dubbo -fix annotation bug] Fix @reference bug (apache#2649) It's fine. * Merge pull request apache#2656, make sure serialization exception sends back to consumer to preventing endless waiting. Fixes apache#1903: Our customized serialization id exceeds the maximum limit, now it cannot work on 2.6.2 anymore. * apache#1903: supplemental change (apache#2666) * Optimize the doSelect method of RandomLoadBalance to reduce the times of invoke of the getWeight method of the AbstractLoadBalance (apache#2597) * 解决与quartz集成报错问题 (apache#2677) quartz初始化的Bean没有beanName,beanName=null,走这段代码的时候报空指针了。 org.springframework.boot.autoconfigure.quartz.AutowireCapableBeanJobFactory类的createJobInstance方法中使用this.beanFactory.initializeBean(jobInstance, null);初始化的Bean,其beanName=null。 * Code format (apache#2662) * NullPointerException * code rule * Merge pull request#2679, consumer throws RpcException when RegistryDirectory notify in high QPS. Fixes apache#2016 * remove ServiceClassHolder and use the ApplicationModel replacedly (apache#2646) * [Dubbo] move the classes of model from config to rpc-api for reusing * remove ServiceClassHolder and use the ApplicationModel replacedly * [Dubbo] delete the unused imported of ApplicationModel * Update dubbo-demo-consumer.xml this commits is no need to commit to remote repo. * [Dubbo-1983] Support Protobuf Serialization (apache#2618) * finish support protobuf * polish * fix code review * use the general test for serialization * Smooth Round Robin selection (apache#2650) * enhance pull request 2618 (apache#2691) * enhance pull request 2618 * move spi file into the right directory * ignore protostuff test case * make unit test pass, support Time type * fix useless imports issue * add license header * fix typo error in FAQ (apache#2684) fix typo error in FAQ * remove author info for RoundRobinLoadBalance (apache#2697) * fix pom for protostuff serialization (apache#2700) * Optimize REGISTRIES field in AbstractRegistryFactory. (apache#2703) * modify ConcurrentHashMap to HashMap.这里REGISTRIES在使用的时候都会先加锁,不存在多线程安全的问题。所以这里建议使用hashmap * modify ConcurrentHashMap to HashMap.这里REGISTRIES在使用的时候都会先加锁,不存在多线程安全的问题。所以这里建议使用hashmap * [version 2.7.0]remove the StaticContext class and refactor the code related to Async (apache#2688) * [Dubbo] move the classes of model from config to rpc-api for reusing * remove ServiceClassHolder and use the ApplicationModel replacedly * [Dubbo] delete the unused imported of ApplicationModel * Update dubbo-demo-consumer.xml this commits is no need to commit to remote repo. * remove StaticContext * remove the unused imported classes of ReferenceConfig * fix NPE of ConsumerModel * fix the failed testcase of AbstractClusterInvokerTest * [Dubbo] #PR2688 to fix the review issues * Optimize heartbeat and reconnect task. (apache#2658) * Optimize heartbeat and reconnect task. 1.Use hashedWheelTimer. 2.Distinguish between reconnect and heartbeat. 3.Increase inspection cycle. * fix ci fail. * fix ci fail. * fix ci fail. * polish the code for pull request 2658: Optimize heartbeat and reconnect task (apache#2709) * 重构 ServiceBean 的 isDelay 方法,使其更符合语义 (apache#2686) * Refactor method isDelay of ServiceBean to fix the wrong meaning of the method * Remove unnecessary method isDelay of ServiceBean * 【Unit Test】FailbackRegistry Test: recover method (apache#2591) * FailbackRegistry Test: recover method * fix the type error, and use CountDownLatch await method to fix the unstable problom * trigger the travis ci test retry * trigger the code static check again * replace ServiceAnnotationBeanPostProcessor log.isInfoEnabled->log.isWarnEnabled (apache#2712) simplify ExtensionLoader code segment * [Dubbo-2678][For Master] Add ability to turn off SPI auto injection, special support for Object type. (apache#2682) * Add ability to turn off SPI auto injection, special support for generic Object type injection. * Change Inject to AutoInject since it's main purpose is to turn off auto-injection. * disable is redundant in DisableInject annotation * Merge pull request apache#2725, problems of graceful shutdown in 2.6.3 and some recommendation. * enhance comments (apache#2735) * code format (apache#2730) * NullPointerException * code rule * code rule * Close all servers/ports after tests finish (apache#2741) * Simplify the code of StringUtils to make it more graceful. (apache#2740) * Simplify the code of StringUtils to make it more graceful. * Add Apache license * Add 'forks' config into xml config (apache#2568) * Add forks into xml config. * Add forks into xml config. * Add forks into xml config. * fix typo (apache#2747) * [DUBBO-2489] MockClusterInvoker provides local forced mock,I tested it locally, but it doesn't work (apache#2742) * apache#2748: Provider should disable mock configuration (apache#2749) * Fix the bug that ReferenceBean refers service more than once when debugging. (apache#2754) note: please visit http://t.cn/EAhta27 for more detail. * Close all servers/ports after tests finish (apache#2755) * add checkstyle rule to check import order (apache#2745) * Changing URL so tests can run in any order (apache#2760) * [Dubbo-2353]fix Invalid property 'interfaceName' of bean class [org.apache.dubbo.config.spring.ServiceBean]apache#2353 (apache#2418) * add getter and setter for ServiceConfig's interfaceName property#2353 * add interfaceName to ignoreAttributeNames and change the unit test * delete the demo source code and update the unit test * unchange ServiceConfig * update unit test * update unit test * apache#2762: [Dubbo - qos-http] stopServer should be invoked ? (apache#2767) * apache#2777: heartbeat threads blocks process shutdown (apache#2778) Fix heartbeat threads blocks process shutdown because its hashed-wheel-timer is non-daemon thread. * oschina vote (apache#2786) * fix typo (apache#2791) Fix typo confilict -> conflict * improvement on Parameters and CollectionUtils (apache#2790) Use CollectionUtils.toStringMap to reduce duplicate code in Parameters, and use Java 8 type inference for collections. * [REFACTOR]:[fix typo] (apache#2804) fix typos * [Dubbo-2798]fix apporiate NotWritablePropertyException (apache#2800) fix apache#2798 . 'propertyname in java donot contain '-' but camel format'. * [Dubbo-2758] Adding dependency on dubbo-remoting-mina for tests in dubbo-rpc-dubbo (apache#2797) * Adding dependency on dubbo-remoting-mina so DubboProtocolTest.testDubboProtocolWithMina is no longer flaky * Changing port number to ensure new server/client created for test * modify dubbo version to 2.6.5 * fix typo of field (apache#2825) * [Dubbo-2845] Fix OOM in UT testSelectByWeight (apache#2845) (apache#2846) Fix OOM Exception generated during UT * Fix UT failed on windows with FileNetworkerTest (apache#2848) Fix FileNetworkerTest failure * Update README.md remove oschina vote * fixed typo of variable (apache#2875) fixed type of variable * Remove unnessesary code in RegistryDirectory#doList (apache#2861) * remove unnessesary code * remove unused import * add some ScriptRoute test case (apache#2879) * add router test case * correct an error word * remove the duplicate superclass of TagRouter and ConditionRouter (apache#2885) Remove the class ConditionRouter and TagRouter implement duplicate superclass Comparable
1 parent 45b3ab4 commit 9ec86f8

File tree

340 files changed

+5258
-9218
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

340 files changed

+5258
-9218
lines changed

FAQ.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,4 @@ For committers, make sure select the right label and target branch for every PR,
2222

2323
[Example question](https://github.com/alibaba/dubbo/issues/742)
2424

25-
Dubbo support to specify ip/port via system environment variables, examples can be found [here](https://github.com/dubbo/dubbo-samples/tree/master/dubbo-samples-docker).
25+
Dubbo supports specifying ip/port via system environment variables, examples can be found [here](https://github.com/dubbo/dubbo-samples/tree/master/dubbo-samples-docker).

PULL_REQUEST_TEMPLATE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ XXXXX
1212

1313
Follow this checklist to help us incorporate your contribution quickly and easily:
1414

15-
- [x] Make sure there is a [GITHUB_issue](https://github.com/apache/incubator-dubbo/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
15+
- [x] Make sure there is a [GITHUB_issue](https://github.com/apache/incubator-dubbo/issues) field for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
1616
- [ ] Format the pull request title like `[Dubbo-XXX] Fix UnknownException when host config not exist #XXX`. Each commit in the pull request should have a meaningful subject line and body.
1717
- [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
1818
- [ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/incubator-dubbo/tree/master/dubbo-test).

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ There's a [README](https://github.com/dubbo/dubbo-samples/blob/master/dubbo-samp
4343
<dependency>
4444
<groupId>com.alibaba</groupId>
4545
<artifactId>dubbo</artifactId>
46-
<version>2.6.4</version>
46+
<version>2.6.5</version>
4747
</dependency>
4848
```
4949

codestyle/checkstyle.xml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,5 +20,11 @@
2020
<module name="NoLineWrap"/>
2121
<module name="OuterTypeFilename"/>
2222
<module name="UnusedImports"/>
23+
24+
<module name="CustomImportOrder">
25+
<property name="specialImportsRegExp" value="org.apache.dubbo.*"/>
26+
<property name="sortImportsInGroupAlphabetically" value="false"/>
27+
<property name="customImportOrderRules" value="SPECIAL_IMPORTS###THIRD_PARTY_PACKAGE###STANDARD_JAVA_PACKAGE###STATIC"/>
28+
</module>
2329
</module>
2430
</module>

codestyle/dubbo_codestyle_for_idea.xml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,6 @@
55
<value>
66
<package name="org.apache.dubbo" withSubpackages="true" static="false"/>
77
<emptyLine/>
8-
<package name="com.taobao" withSubpackages="true" static="false"/>
9-
<emptyLine/>
108
<package name="" withSubpackages="true" static="false"/>
119
<emptyLine/>
1210
<package name="javax" withSubpackages="true" static="false"/>

dubbo-all/pom.xml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -320,7 +320,7 @@
320320
</dependency>
321321
<dependency>
322322
<groupId>org.apache.dubbo</groupId>
323-
<artifactId>dubbo-bootstrap</artifactId>
323+
<artifactId>dubbo-serialization-protostuff</artifactId>
324324
<version>${project.version}</version>
325325
<scope>compile</scope>
326326
<optional>true</optional>
@@ -431,7 +431,7 @@
431431
<include>org.apache.dubbo:dubbo-serialization-fst</include>
432432
<include>org.apache.dubbo:dubbo-serialization-kryo</include>
433433
<include>org.apache.dubbo:dubbo-serialization-jdk</include>
434-
<include>org.apache.dubbo:dubbo-bootstrap</include>
434+
<include>org.apache.dubbo:dubbo-serialization-protostuff</include>
435435
</includes>
436436
</artifactSet>
437437
<transformers>
@@ -595,4 +595,4 @@
595595
</build>
596596
</profile>
597597
</profiles>
598-
</project>
598+
</project>

dubbo-bom/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@
280280
</dependency>
281281
<dependency>
282282
<groupId>org.apache.dubbo</groupId>
283-
<artifactId>dubbo-bootstrap</artifactId>
283+
<artifactId>dubbo-serialization-protostuff</artifactId>
284284
<version>${project.version}</version>
285285
</dependency>
286286
<dependency>

dubbo-bootstrap/src/main/java/org/apache/dubbo/bootstrap/DubboBootstrap.java

Lines changed: 0 additions & 111 deletions
This file was deleted.

dubbo-cluster/src/main/java/org/apache/dubbo/rpc/cluster/loadbalance/RandomLoadBalance.java

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,13 +34,13 @@ public class RandomLoadBalance extends AbstractLoadBalance {
3434
@Override
3535
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
3636
int length = invokers.size(); // Number of invokers
37-
int totalWeight = 0; // The sum of weights
3837
boolean sameWeight = true; // Every invoker has the same weight?
39-
for (int i = 0; i < length; i++) {
38+
int firstWeight = getWeight(invokers.get(0), invocation);
39+
int totalWeight = firstWeight; // The sum of weights
40+
for (int i = 1; i < length; i++) {
4041
int weight = getWeight(invokers.get(i), invocation);
4142
totalWeight += weight; // Sum
42-
if (sameWeight && i > 0
43-
&& weight != getWeight(invokers.get(i - 1), invocation)) {
43+
if (sameWeight && weight != firstWeight) {
4444
sameWeight = false;
4545
}
4646
}

dubbo-cluster/src/main/java/org/apache/dubbo/rpc/cluster/loadbalance/RoundRobinLoadBalance.java

Lines changed: 111 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -17,68 +17,137 @@
1717
package org.apache.dubbo.rpc.cluster.loadbalance;
1818

1919
import org.apache.dubbo.common.URL;
20-
import org.apache.dubbo.common.utils.AtomicPositiveInteger;
2120
import org.apache.dubbo.rpc.Invocation;
2221
import org.apache.dubbo.rpc.Invoker;
2322

24-
import java.util.ArrayList;
23+
import java.util.Collection;
24+
import java.util.Iterator;
2525
import java.util.List;
26+
import java.util.Map;
27+
import java.util.Map.Entry;
2628
import java.util.concurrent.ConcurrentHashMap;
2729
import java.util.concurrent.ConcurrentMap;
30+
import java.util.concurrent.atomic.AtomicBoolean;
31+
import java.util.concurrent.atomic.AtomicLong;
2832

2933
/**
3034
* Round robin load balance.
3135
*/
3236
public class RoundRobinLoadBalance extends AbstractLoadBalance {
33-
3437
public static final String NAME = "roundrobin";
38+
39+
private static int RECYCLE_PERIOD = 60000;
40+
41+
protected static class WeightedRoundRobin {
42+
private int weight;
43+
private AtomicLong current = new AtomicLong(0);
44+
private long lastUpdate;
45+
public int getWeight() {
46+
return weight;
47+
}
48+
public void setWeight(int weight) {
49+
this.weight = weight;
50+
current.set(0);
51+
}
52+
public long increaseCurrent() {
53+
return current.addAndGet(weight);
54+
}
55+
public void sel(int total) {
56+
current.addAndGet(-1 * total);
57+
}
58+
public long getLastUpdate() {
59+
return lastUpdate;
60+
}
61+
public void setLastUpdate(long lastUpdate) {
62+
this.lastUpdate = lastUpdate;
63+
}
64+
}
3565

36-
private final ConcurrentMap<String, AtomicPositiveInteger> sequences = new ConcurrentHashMap<String, AtomicPositiveInteger>();
37-
38-
private final ConcurrentMap<String, AtomicPositiveInteger> indexSeqs = new ConcurrentHashMap<String, AtomicPositiveInteger>();
39-
66+
private ConcurrentMap<String, ConcurrentMap<String, WeightedRoundRobin>> methodWeightMap = new ConcurrentHashMap<String, ConcurrentMap<String, WeightedRoundRobin>>();
67+
private AtomicBoolean updateLock = new AtomicBoolean();
68+
69+
/**
70+
* get invoker addr list cached for specified invocation
71+
* <p>
72+
* <b>for unit test only</b>
73+
*
74+
* @param invokers
75+
* @param invocation
76+
* @return
77+
*/
78+
protected <T> Collection<String> getInvokerAddrList(List<Invoker<T>> invokers, Invocation invocation) {
79+
String key = invokers.get(0).getUrl().getServiceKey() + "." + invocation.getMethodName();
80+
Map<String, WeightedRoundRobin> map = methodWeightMap.get(key);
81+
if (map != null) {
82+
return map.keySet();
83+
}
84+
return null;
85+
}
86+
4087
@Override
4188
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
4289
String key = invokers.get(0).getUrl().getServiceKey() + "." + invocation.getMethodName();
43-
int length = invokers.size(); // Number of invokers
44-
int maxWeight = 0; // The maximum weight
45-
int minWeight = Integer.MAX_VALUE; // The minimum weight
46-
final List<Invoker<T>> nonZeroWeightedInvokers = new ArrayList<>();
47-
for (int i = 0; i < length; i++) {
48-
int weight = getWeight(invokers.get(i), invocation);
49-
maxWeight = Math.max(maxWeight, weight); // Choose the maximum weight
50-
minWeight = Math.min(minWeight, weight); // Choose the minimum weight
51-
if (weight > 0) {
52-
nonZeroWeightedInvokers.add(invokers.get(i));
53-
}
54-
}
55-
AtomicPositiveInteger sequence = sequences.get(key);
56-
if (sequence == null) {
57-
sequences.putIfAbsent(key, new AtomicPositiveInteger());
58-
sequence = sequences.get(key);
90+
ConcurrentMap<String, WeightedRoundRobin> map = methodWeightMap.get(key);
91+
if (map == null) {
92+
methodWeightMap.putIfAbsent(key, new ConcurrentHashMap<String, WeightedRoundRobin>());
93+
map = methodWeightMap.get(key);
5994
}
60-
61-
if (maxWeight > 0 && minWeight < maxWeight) {
62-
AtomicPositiveInteger indexSeq = indexSeqs.get(key);
63-
if (indexSeq == null) {
64-
indexSeqs.putIfAbsent(key, new AtomicPositiveInteger(-1));
65-
indexSeq = indexSeqs.get(key);
95+
int totalWeight = 0;
96+
long maxCurrent = Long.MIN_VALUE;
97+
long now = System.currentTimeMillis();
98+
Invoker<T> selectedInvoker = null;
99+
WeightedRoundRobin selectedWRR = null;
100+
for (Invoker<T> invoker : invokers) {
101+
String identifyString = invoker.getUrl().toIdentityString();
102+
WeightedRoundRobin weightedRoundRobin = map.get(identifyString);
103+
int weight = getWeight(invoker, invocation);
104+
if (weight < 0) {
105+
weight = 0;
66106
}
67-
length = nonZeroWeightedInvokers.size();
68-
while (true) {
69-
int index = indexSeq.incrementAndGet() % length;
70-
int currentWeight;
71-
if (index == 0) {
72-
currentWeight = sequence.incrementAndGet() % maxWeight;
73-
} else {
74-
currentWeight = sequence.get() % maxWeight;
75-
}
76-
if (getWeight(nonZeroWeightedInvokers.get(index), invocation) > currentWeight) {
77-
return nonZeroWeightedInvokers.get(index);
107+
if (weightedRoundRobin == null) {
108+
weightedRoundRobin = new WeightedRoundRobin();
109+
weightedRoundRobin.setWeight(weight);
110+
map.putIfAbsent(identifyString, weightedRoundRobin);
111+
weightedRoundRobin = map.get(identifyString);
112+
}
113+
if (weight != weightedRoundRobin.getWeight()) {
114+
//weight changed
115+
weightedRoundRobin.setWeight(weight);
116+
}
117+
long cur = weightedRoundRobin.increaseCurrent();
118+
weightedRoundRobin.setLastUpdate(now);
119+
if (cur > maxCurrent) {
120+
maxCurrent = cur;
121+
selectedInvoker = invoker;
122+
selectedWRR = weightedRoundRobin;
123+
}
124+
totalWeight += weight;
125+
}
126+
if (!updateLock.get() && invokers.size() != map.size()) {
127+
if (updateLock.compareAndSet(false, true)) {
128+
try {
129+
// copy -> modify -> update reference
130+
ConcurrentMap<String, WeightedRoundRobin> newMap = new ConcurrentHashMap<String, WeightedRoundRobin>();
131+
newMap.putAll(map);
132+
Iterator<Entry<String, WeightedRoundRobin>> it = newMap.entrySet().iterator();
133+
while (it.hasNext()) {
134+
Entry<String, WeightedRoundRobin> item = it.next();
135+
if (now - item.getValue().getLastUpdate() > RECYCLE_PERIOD) {
136+
it.remove();
137+
}
138+
}
139+
methodWeightMap.put(key, newMap);
140+
} finally {
141+
updateLock.set(false);
78142
}
79143
}
80144
}
81-
// Round robin
82-
return invokers.get(sequence.getAndIncrement() % length);
145+
if (selectedInvoker != null) {
146+
selectedWRR.sel(totalWeight);
147+
return selectedInvoker;
148+
}
149+
// should not happen here
150+
return invokers.get(0);
83151
}
152+
84153
}

0 commit comments

Comments
 (0)