Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

imbajin
Copy link
Contributor

@imbajin imbajin commented Apr 30, 2019

Need more Unit-Tests for different update strategy.

MultivaluedHashMap<String, Object> headers = new MultivaluedHashMap<>();
headers.putSingle("Content-Encoding", BATCH_ENCODING);
String updateStrategies = GraphAPI.formatProperties(strategies);
Map<String ,Object> params = ImmutableMap.of("updateStrategies",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updateStrategies -> update_strategies

return ids;
}

public List<Edge> update(List<Edge> edges,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modify according to server comments

updateStrategies,
"createIfNotExist",
createIfNotExist);
RestResult result = this.client.post(this.batchPath() + "Update",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this.client.put(this.batchPath(), ...)

return ids;
}

public List<Vertex> update(List<Vertex> vertices,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add api test cases for each strategy

@imbajin imbajin force-pushed the support-batch-update branch from 9c6140d to 99558b2 Compare July 10, 2019 03:06

// TODO: Edge seems OK, should we keep the JsonEdge's code?
@JsonProperty("edges")
private List<Edge> jsonEdges;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Edge is enough

}

@JsonIgnoreProperties(value = {"type"})
private class JsonEdge {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete it

"Parameter 'update_strategies' cannot be empty");
// Not support createIfNotExist equals false now
E.checkArgument(req.createIfNotExist == true, "Parameter " +
"'create_if_not_exist' is not supported now");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is not supported false now

public class BatchVertexRequest {

// TODO: Vertex seems OK, should we keep the JsonVertex's code?
@JsonProperty("vertices")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vertex is enough


public Builder() {
this.req = new BatchVertexRequest();
//this.vertexBuilders = new ArrayList<>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems open the comment is better

}

public BatchVertexRequest build() {
/*this.vertexBuilders.forEach(builder -> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it need?

package com.baidu.hugegraph.api.graph.structure;

public enum UpdateStrategy {
SUM,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add an empty line

Object price = vertex.properties().get("price");
Assert.assertTrue(price instanceof Number);
Assert.assertTrue((int) price == 0);
System.out.println(vertex);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove System.out.println(vertex)

this.graph().updateVertices(req).forEach(vertex -> {
Object price = vertex.properties().get("price");
Assert.assertTrue(price instanceof Number);
Assert.assertTrue((int) price == 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Assert.assertEquals


@Test
public void testBatchUpdateStrategySmaller() {
BatchVertexRequest req = batchVertexRequest("fullDate", -1, 1,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use fullDate(Date property), it should actually be deserialized to date, now being deserialized to number is a bug

imbajin and others added 5 commits July 10, 2019 11:50
@imbajin imbajin force-pushed the support-batch-update branch from f47a393 to a2b555a Compare July 10, 2019 06:55
BatchVertexRequest req = batchVertexRequest("price", 1, -1,
UpdateStrategy.SUM);

this.graph().updateVertices(req).forEach(vertex -> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call vertexAPI.update in api test

this.graph().updateVertices(req).forEach(vertex -> {
Object price = vertex.properties().get("price");
Assert.assertTrue(price instanceof Number);
Assert.assertTrue((int) price > 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assert.assertEquals(1, price);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable i is now appended to the test data, so the attribute of each element are not fixed values.
And due to the order of the collection is not fixed from server's response, cannot be one-to-one correspondence with index, like set-property, price-property, and others.
So shall we get rid of all the changes so that elements can have the same property values?

this.graph().updateVertices(req).forEach(vertex -> {
Object price = vertex.properties().get("price");
Assert.assertTrue(price instanceof Number);
Assert.assertTrue((int) price < 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assert.assertEquals(-1, price);

this.graph().updateVertices(req).forEach(vertex -> {
Object set = vertex.properties().get("set");
Assert.assertTrue(set instanceof Collection);
Assert.assertEquals(2, ((Collection) set).size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add Assert.assertEquals(ImmutableSet.of("xx", "xx"), set);
same as other tests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Collection<?>) set

graph().updateEdges(req).forEach(edge -> {
Object set = edge.properties().get("set");
Assert.assertTrue(set instanceof Collection);
Assert.assertTrue(((Collection)set).size() == 2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

        Assert.assertTrue(((Collection<?>) set).size() == 2);

Object set = edge.properties().get("set");
Assert.assertTrue(set instanceof Collection);
Assert.assertTrue(((Collection)set).size() == 2);
System.out.println(edge);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove it

this.graph().updateVertices(req).forEach(vertex -> {
Object list = vertex.properties().get("list");
Assert.assertTrue(list instanceof List);
Assert.assertEquals(2, ((List) list).size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

((List<?>) list)

this.graph().updateVertices(req).forEach(vertex -> {
Object list = vertex.properties().get("list");
Assert.assertTrue(list instanceof List);
Assert.assertTrue(((List) list).isEmpty());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

((List<?>) list)

this.graph().updateVertices(req).forEach(vertex -> {
Object set = vertex.properties().get("set");
Assert.assertTrue(set instanceof Collection);
Assert.assertTrue(((Collection) set).isEmpty());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Collection<?>) set

this.graph().updateVertices(req).forEach(vertex -> {
Object set = vertex.properties().get("set");
Assert.assertTrue(set instanceof Collection);
Assert.assertEquals(2, ((Collection) set).size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Collection<?>) set


// TODO: Consider better way to check exception & deserialized result
int responseCode = result.status();
if (responseCode != 200) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checked by RestClient.checkStatus(), just remove this check


// TODO: Consider better way to check exception & deserialized result
int responseCode = result.status();
if (responseCode != 200) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

public void testBatchUpdateStrategyBigger() {
BatchEdgeRequest req = batchEdgeRequest("price", 1, -1,
UpdateStrategy.BIGGER);
this.edgeAPI.update(req).forEach(edge -> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert response size equals to batch size

private BatchEdgeRequest batchEdgeRequest(String key, Object oldData,
Object newData,
UpdateStrategy strategy) {
Map<String, UpdateStrategy> strategies = ImmutableMap.of(key, strategy);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer to move to line 683

return this;
}

public Builder updateStrategies(Map<String, UpdateStrategy> maps) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename method name to updatingStrategies
rename maps to map or strategies

}

public Builder updateStrategies(Map<String, UpdateStrategy> maps) {
this.req.updateStrategies = maps;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new Map(maps) to clone

public Builder updateStrategies(Map<String, UpdateStrategy> maps) {
this.req.updateStrategies = maps;
return this;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add method:

       public Builder updateStrategy(String property, UpdateStrategy strategy) {
           updateStrategies.put(property, strategy);
       }

Assert.assertTrue(((List<?>) list).isEmpty());
});
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add some negative tests

schema.propertyKey("price").asInt().ifNotExist().create();
schema.propertyKey("weight").asDouble().ifNotExist().create();
// Add for assertBatchResponse updatingStrategies & Date type
schema.propertyKey("fullDate").asDate().ifNotExist().create();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change propertyKey("date").asText() to propertyKey("date").asDate()

.ifNotExist()
.create();

schema.vertexLabel("testV")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need a specific name, it represents a scene

.ifNotExist()
.create();

schema.edgeLabel("testE")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

.create();

schema.vertexLabel("testV")
.properties("name", "set", "fullDate", "price", "list")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please keep order according to a certain rule

schema.vertexLabel("testV")
.properties("name", "set", "fullDate", "price", "list")
.primaryKeys("name")
.nullableKeys("set", "fullDate", "price", "list")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

this.assertBatchResponse(vertices, "price", 5);
}

// TODO: Add date comparison after fixing the date serialization bug
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start with /**

@codecov
Copy link

codecov bot commented Jul 16, 2019

Codecov Report

Merging #46 into master will increase coverage by 0.09%.
The diff coverage is 78.37%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master      #46      +/-   ##
============================================
+ Coverage     74.96%   75.06%   +0.09%     
- Complexity      502      508       +6     
============================================
  Files            91       94       +3     
  Lines          2209     2282      +73     
  Branches        137      143       +6     
============================================
+ Hits           1656     1713      +57     
- Misses          452      466      +14     
- Partials        101      103       +2
Impacted Files Coverage Δ Complexity Δ
...baidu/hugegraph/api/traverser/NeighborRankAPI.java 85.29% <ø> (ø) 3 <0> (ø) ⬇️
.../baidu/hugegraph/structure/schema/PropertyKey.java 68% <ø> (ø) 3 <0> (ø) ⬇️
.../java/com/baidu/hugegraph/driver/GraphManager.java 77.27% <0%> (-3.68%) 52 <0> (ø)
.../hugegraph/api/graph/structure/UpdateStrategy.java 100% <100%> (ø) 1 <1> (?)
.../main/java/com/baidu/hugegraph/util/TaskCache.java 84.44% <100%> (ø) 17 <0> (ø) ⬇️
.../java/com/baidu/hugegraph/api/graph/VertexAPI.java 93.02% <100%> (+0.71%) 11 <1> (+1) ⬆️
...in/java/com/baidu/hugegraph/api/graph/EdgeAPI.java 93.47% <100%> (+0.62%) 11 <1> (+1) ⬆️
...egraph/api/graph/structure/BatchVertexRequest.java 79.16% <79.16%> (ø) 1 <1> (?)
...ugegraph/api/graph/structure/BatchEdgeRequest.java 81.48% <81.48%> (ø) 1 <1> (?)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c6fb43...e807719. Read the comment docs.

.primaryKeys("name")
.nullableKeys("price", "date", "set", "list")
.ifNotExist()
.create();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to BatchUpdateElementApiTest

.properties("name", "price", "date", "set", "list")
.nullableKeys("name", "price", "date", "set", "list")
.ifNotExist()
.create();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to BatchUpdateElementApiTest

data[1] + index), value);
}
});
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to BatchUpdateElementApiTest


public class BatchUpdateElementApiTest extends BaseApiTest {

final int number = 5;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to:
private static final int BATCH_SIZE = 5;

@Test
public void testVertexEmptyUpdateStrategy() {
List<Vertex> vertices = this.createNVertexBatch("object", "new", 5);
Map<String, UpdateStrategy> strategies = Collections.emptyMap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ImmutableMap.of()


@Test
public void testVertexEmptyUpdateStrategy() {
List<Vertex> vertices = this.createNVertexBatch("object", "new", 5);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mock instead

edge.sourceLabel("object");
edge.targetId("object:2");
edge.targetLabel("object");
edge.property("price", 1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If these code does not affect the test logic, we can add a method createEdge()

req = batchVertexRequest("list", "old", "x", UpdateStrategy.ELIMINATE);
vertices = this.vertexAPI.update(req);
this.assertBatchResponse(vertices, "list", "old");
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add negative test like updateStrategies is null or empty:

Whitebox.setInternalState(req, "updateStrategies", empty); 
this.vertexAPI.update(req); // expect exception

.createIfNotExist(true)
.build();
});
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a test that build ok

return req;
}

private void assertBatchResponse(List<? extends GraphElement> elements,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set static

});
}

private void assertBatchResponse(List<? extends GraphElement> elements,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set static

Assert.assertEquals(true, created);
}

private Vertex createVertex() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set static

return vertex;
}

private Edge createEdge() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set static

import com.google.common.collect.ImmutableMap;

public class BatchElementRequestTest extends BaseClientTest {
public class BatchElementRequestTest {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extends BaseUnitTest
and add it to UnitTestSuite

edges.add(edge);
}
return edges;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also move to BatchUpdateElementApiTest

Assert.assertThrows(ServerException.class, () -> {
this.edgeAPI.update(req);
});
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add test cases:

 updateStrategies is empty map
 createIfNotExist = false
 edges is null
 edges is empty
 one of edge in edges is null

}

@Test
public void testEdgeEmptyUpdateStrategy() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to testEdgeBatchUpdateWithInvalidArgs()

}

@Test
public void testVertexRequestBuildOK() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to line 41

}

@Test
public void testEdgeRequestBuildOK() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to line 89

import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;

public class BatchUpdateElementApiTest extends BaseApiTest {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

register BatchUpdateElementApiTest to com.baidu.hugegraph.api.ApiTestSuite

Assert.assertThrows(ServerException.class, () -> {
Whitebox.setInternalState(req, "vertices", null);
vertexAPI.update(req);
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some tests may not return the expected exception, so please add assert on the exception message:

        Assert.assertThrows(ServerException.class, () -> {
            ...;
        }, e -> {
            String expect = "...";
            Assert.assertTrue(e.toString(), e.getMessage().contains(expect));
        });

the same as other tests


Assert.assertThrows(ServerException.class, () -> {
Whitebox.setInternalState(req, "vertices", ImmutableList.of());
vertexAPI.update(req);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems the server will not check vertices is empty.

Assert.assertThrows(ServerException.class, () -> {
Whitebox.setInternalState(req, "vertices", ImmutableList.of());
vertexAPI.update(req);
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add cases:

  • vertices.size() > 500
  • property type error like SUM strategy with string property, or UNION strategy with string property. (add a new test method for this cases)
  • other exception cases

}

public BatchVertexRequest build() {
E.checkArgumentNotNull(req, "BatchVertexRequest cannot be null");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer to replace all "cannot" to "can't"

return this;
}

public Builder updatingStrategies(Map<String, UpdateStrategy> map) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updateStrategies

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refer to here
seems to distinguish between updating strategies (nouns) & update some strategies (verb)

return this;
}

public Builder updatingStrategy(String property,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

return this;
}

public Builder updatingStrategies(Map<String, UpdateStrategy> map) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

return this;
}

public Builder updatingStrategy(String property,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

"Parameter 'update_strategies' can't be empty");
E.checkArgument(req.createIfNotExist == true,
"Parameter 'create_if_not_exist' " +
"dose not supported false now");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does not support false now

vertexAPI.update(req1);
}, e -> {
String expect = "Property must be Number, but got String, String";
String expect = "Property must be Number type for strategy SUM, " +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Property type must be Number

javeme
javeme previously approved these changes Jul 23, 2019
vertexAPI.update(req6);
}, e -> {
String expect = "Property type must be Set or List for " +
"strategy UNION, but got type Integer, Integer";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPEND

}

public List<Edge> update(BatchEdgeRequest request) {
RestResult result = this.client.put(this.batchPath(), null, request);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pass headers

MultivaluedHashMap<String, Object> headers = new MultivaluedHashMap<>();
headers.putSingle("Content-Encoding", BATCH_ENCODING);

}

public List<Vertex> update(BatchVertexRequest request) {
RestResult result = this.client.put(this.batchPath(), null, request);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@imbajin imbajin force-pushed the support-batch-update branch from 4dbe08b to 7b2f16b Compare July 24, 2019 06:56
@javeme javeme merged commit 1dc9e47 into hugegraph:master Jul 25, 2019
zhoney pushed a commit to zhoney/hugegraph-client that referenced this pull request Sep 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants