StyleClassedTextArea stuck when appending 100,000 lines one by one.

[hrzafer]

I just added the following method to ManualHighlighting.java and call at the end of start(Stage primaryStage):

private void testLongText(){

    Map<Integer, String> styles = new HashMap<>();
    styles.put(0, "red");
    styles.put(1, "green");
    styles.put(2, "blue");

    String text = "This is a test\n";
    int times = 1000;
    for (int i=0; i<times; i++){
        area.replaceText(area.getLength(), area.getLength(), text);
        area.setStyleClass(area.getLength() - text.length(), area.getLength(), styles.get(i%3) );
    }
    area.requestFocus();
}

Here is my style.css:

.red {
    -fx-font-size: 12pt;
    -fx-fill: "red";
}

.green {
    -fx-font-size: 12pt;
    -fx-fill: "green";
}

.blue {
    -fx-font-size: 12pt;
    -fx-fill: "blue";
}

This is what I get with 155 MB memory consumption:

When I set times = 10000; it stuck for 3-4 seconds and then I can see the same window above. The memory consumption is 475 MB. By the way a text file with 10000 lines of This is a test\n is 156 KB.

When I set times = 100000; the window does not respond.

pom.xml:

<dependency>
  <groupId>org.fxmisc.richtext</groupId>
  <artifactId>richtextfx</artifactId>
  <version>0.6.2</version>
</dependency>

Hi just curious. How would a regular java text area compare? It might help
to measure against a reference.
On Feb 25, 2015 1:22 AM, "Harun Reşit Zafer" notifications@github.com
wrote:

I just added the following method to ManualHighlighting.java and call at
the end of start(Stage primaryStage):

private void testLongText(){

Map<Integer, String> styles = new HashMap<>();
styles.put(0, "red");
styles.put(1, "green");
styles.put(2, "blue");

String text = "This is a test\n";
int times = 1000;
for (int i=0; i<times; i++){
    area.replaceText(area.getLength(), area.getLength(), text);
    area.setStyleClass(area.getLength() - text.length(), area.getLength(), styles.get(i%3) );
}
area.requestFocus();

}

Here is my style.css:

.red {
-fx-font-size: 12pt;
-fx-fill: "red";
}

.green {
-fx-font-size: 12pt;
-fx-fill: "green";
}

.blue {
-fx-font-size: 12pt;
-fx-fill: "blue";
}

This is what I get with 155 MB memory consumption:

[image: capture]
https://cloud.githubusercontent.com/assets/4048197/6367814/79451af4-bcdf-11e4-8a15-2429f366fbe3.PNG

When I set times = 10000; it stuck for 3-4 seconds and then I can see the
same window above. The memory consumption is 475 MB. By the way a text file
with 10000 lines of This is a test\n is 156 KB.

When I set times = 100000; the window does not respond.

pom.xml:

org.fxmisc.richtext richtextfx 0.6.2

—
Reply to this email directly or view it on GitHub
#121.

[TomasMikula]

@melkhaldi yes, that is a good point. We should also compare against a ListView of TextFlows (one TextFlow for each line), since that is closer to how RichTextFX is implemented.

@hrzafer Can you confirm this is with the 0.6.2 version? (EDIT: never mind, I see you included part of your pom.xml)

[TomasMikula]

I edited the title, because the long text is not really the issue, but rather the repeated replceText/setStyleClass.

When you are inserting text at position i, the current implementation to find out the line for i is linear in the number of lines. So if you do this 100,000 times with 100,000 lines, this means ~10,000,000,000 operations (where by operation I don't mean processor instructions, but something more complex, where each "operation" can take thousands of processor instructions).

This can be optimized to be logarithmic instead of linear.

Do you have a real world use where you do something like this, though?
Are you interested in a workaround? That would be constructing a StyledDocument and using replace(from, to, styledDocument).

[hrzafer]

I'm developing a demo for our language identifier. We want each text block (sentence, paragraph etc.) in a different color related with its language. We also want the text area editable to be able to enter text manually. I'll try EditableStyledDocument instead of StyledDocument. There is not an example on the readme file though.

[TomasMikula]

EditableStyledDocument is private. Anyway, it is the one that has the inefficient implementation I mentioned above.

You need to create your own implementation of StyledDocument interface, construct it efficiently and then use to update the text area.

Alternatively, you can construct one String with all the text, and one instance of StyleSpans, which defines style ranges, without the text, and then use just one replaceText and one setStyleSpans call.

The API is going to improve, it will be easier to work with StyledDocuments, but if you need something to work now, these are your options.

[hrzafer]

Alternatively, you can construct one String with all the text, and one instance of StyleSpans, which defines style ranges, without the text, and then use just one replaceText and one setStyleSpans call.

Sounds like a better work around, I'll try this and give you feedback ASAP.

[hrzafer]

I couldn't figure out how to do that, the following code does not work.

String text = "This is a test\n";
int times = 10;
StringBuilder sb = new StringBuilder();
StyleSpans<Collection<String>> styleSpans = area.getStyleSpans(0, area.getLength());
for (int i=0; i<times; i++){
    sb.append(text);
    styleSpans.append(Arrays.asList(new String[]{styles.get(i % 3)}), text.length());
    styleSpans.append(Arrays.asList(new String[] {}), 1);//For the newline I suppose        
}

area.replaceText(area.getLength(), area.getLength(), sb.toString());
area.setStyleSpans(0, styleSpans);

[TomasMikula]

styleSpans.append does not append to the current StyleSpans, but returns a new one. Also, when the newline is already in the string text, then style span of length text.length() already covers the newline.

A more efficient way than appending style spans (which creates a new instance every time), however, is to use StyleSpansBuilder with an initial capacity:

private void testLongText(){
    List<Collection<String>> styles = new ArrayList<>(3);
    styles.add(Collections.singleton("red"));
    styles.add(Collections.singleton("green"));
    styles.add(Collections.singleton("blue"));

    String text = "This is a test\n";
    int times = 3000;
    StringBuilder sb = new StringBuilder();
    StyleSpansBuilder<Collection<String>> ssb = new StyleSpansBuilder<>(times);
    for (int i=0; i<times; i++) {
        sb.append(text);
        ssb.add(styles.get(i%3), text.length());
    }

    int oldLength = area.getLength();
    area.appendText(sb.toString());
    area.setStyleSpans(oldLength, ssb.create());

    area.requestFocus();
}

[hrzafer]

Thank you very much for the help. It works a lot faster now.

[hwaite]

I'm trying to format a 10,000 line .diff file with roughly 2500 style regions and am running into stack overflow errors. Not sure if this is a distinct problem or not.

Exception in thread "JavaFX Application Thread" java.lang.StackOverflowError
  at java.util.Collections$UnmodifiableCollection.size(Collections.java:1030)
  at org.reactfx.util.ListsHelper$ConcatListView.size(Lists.java:250)
  at java.util.Collections$UnmodifiableCollection.size(Collections.java:1030)
  at org.reactfx.util.ListsHelper$ConcatListView.size(Lists.java:250)
  ...

Relevant code below:

Set<String> style = null;
int length = 0;
final StyleSpansBuilder<Collection<String>> spansBuilder = new StyleSpansBuilder<>();
for (String line : diffLines) {
    final Set<String> newStyle =
     line.startsWith("---") ? NONE :
     line.startsWith("+++") ? NONE :
     line.startsWith("@@") ? HEADER :
     line.startsWith("+") ? ADDITION :
     line.startsWith("-") ? REMOVAL :
     REGULAR;

    // first element
    if (length == 0) {
        style = newStyle;

    // new style
    } else if (!Objects.equals(style, newStyle)) {
        spansBuilder.add(style, length);
        length = 0;
        style = newStyle;
    }

    length += line.length() + LINE_SEPARATOR.length();
}

// final element
if (length > 0) spansBuilder.add(style, length);

mCodeArea.clear();
mCodeArea.appendText(String.join(LINE_SEPARATOR, diffLines) + LINE_SEPARATOR);
mCodeArea.setStyleSpans(0, spansBuilder.create());

Problem seems to be that ListsHelper.ConcatListView is populating in a highly nested/inefficient manner. List of size n comprises two component lists, both of which are decorated by Collections.UnmodifiableList. The first list is another ConcatListView of size n - 1 while the latter is a normal Paragraph element.

at org.reactfx.util.ListsHelper$ConcatListView.<init>(Lists.java:236)
at org.reactfx.util.ListsHelper.concatView(Lists.java:226)
at org.reactfx.util.ListsHelper.concatView(Lists.java:205)
at org.reactfx.util.Lists.concatView(Lists.java:197)
at org.reactfx.util.Lists.concatView(Lists.java:190)
at org.reactfx.collection.ListChangeAccumulator.combine(ListChangeAccumulator.java:154)
at org.reactfx.collection.ListChangeAccumulator.add(ListChangeAccumulator.java:80)
at org.reactfx.collection.ListChangeAccumulator.add(ListChangeAccumulator.java:91)
at org.reactfx.util.AccumulationFacility$ListChangeAccumulation.reduce(AccumulationFacility.java:66)
at org.reactfx.util.AccumulationFacility$ListChangeAccumulation.reduce(AccumulationFacility.java:53)
at org.reactfx.SuspendableBase.reduce(SuspendableBase.java:34)
at org.reactfx.SuspendableBase.handleEvent(SuspendableBase.java:76)
at org.reactfx.SuspendableBase$$Lambda$178.1236970722.accept(Unknown Source:-1)
at org.reactfx.util.NonAccumulativeStreamNotifications.lambda$head$1(NotificationAccumulator.java:135)
at org.reactfx.util.NonAccumulativeStreamNotifications$$Lambda$674.512461004.run(Unknown Source:-1)
at org.reactfx.ObservableBase.notifyObservers(ObservableBase.java:68)
at org.reactfx.ObservableBase.notifyObservers(ObservableBase.java:57)
at org.reactfx.ProperEventStream.emit(ProperEventStream.java:18)
at org.reactfx.collection.LiveList$3$$Lambda$441.1670669293.onChange(Unknown Source:-1)
at org.reactfx.collection.LiveList.lambda$observeQuasiChanges$86(LiveList.java:256)
at org.reactfx.collection.LiveList$$Lambda$442.1470779585.onChanged(Unknown Source:-1)
at com.sun.javafx.collections.ListListenerHelper$SingleChange.fireValueChangedEvent(ListListenerHelper.java:164)
at com.sun.javafx.collections.ListListenerHelper.fireValueChangedEvent(ListListenerHelper.java:73)
at javafx.collections.ObservableListBase.fireChange(ObservableListBase.java:233)
at javafx.collections.FXCollections$UnmodifiableObservableListImpl.lambda$new$52(FXCollections.java:929)
at javafx.collections.FXCollections$UnmodifiableObservableListImpl$$Lambda$42.1849707334.onChanged(Unknown Source:-1)
at javafx.collections.WeakListChangeListener.onChanged(WeakListChangeListener.java:88)
at com.sun.javafx.collections.ListListenerHelper$Generic.fireValueChangedEvent(ListListenerHelper.java:329)
at com.sun.javafx.collections.ListListenerHelper.fireValueChangedEvent(ListListenerHelper.java:73)
at javafx.collections.ObservableListBase.fireChange(ObservableListBase.java:233)
at javafx.collections.ListChangeBuilder.commit(ListChangeBuilder.java:482)
at javafx.collections.ListChangeBuilder.endChange(ListChangeBuilder.java:541)
at javafx.collections.ObservableListBase.endChange(ObservableListBase.java:205)
at javafx.collections.ModifiableObservableListBase.set(ModifiableObservableListBase.java:163)
at org.fxmisc.richtext.EditableStyledDocument.setStyleSpans(EditableStyledDocument.java:315)
at org.fxmisc.richtext.StyledTextArea.setStyleSpans(StyledTextArea.java:651)

The ReactFX code is pretty complicated; I can't easily determine what you're trying to accomplish here. Is a more efficient implementation feasible? Retrieving elements or even just calling size() on these LISP-ish structures seems unnecessarily expensive.

[TomasMikula]

While this might add to the performance cost, this is a different problem than what I described above (the linear cost of looking up a paragraph for the given text offset), so please open a separate issue. Thanks.

[TomasMikula]

The current snapshot now optimizes lookups of absolute position to be logarithmic in the number of paragraphs.

The test case with 100,000 edits at once still takes about 45s to load on my laptop.

I tried the same, minus setStyleClass, with the JavaFX's TextArea (i.e. just appending a line of text 100,000 times) and I gave up after 5 minutes of waiting.

Since the asymptotic complexity has been optimized, the performance is much better than JavaFX's TextArea, and there was a better solution from the start, I am closing this issue.