Frequently Asked Questions

1. Is TopoART a deep neural network?

No, at least not in the classic sense. A single TopoART module contains just three layers of nodes which interact differently from the layers of networks commonly used in deep learning. Additional TopoART modules act as a kind of input filter for subsequent modules. This enables TopoART to group its input at different levels of granularity, but it does not compute feature maps at different levels of abstraction. This is a major difference from current deep neural networks. As a result, TopoART may use many node layers depending on its number of modules, but it still cannot be considered a deep neural network.

2. Can TopoART be combined with deep neural networks?

Yes. A deep neural network can serve as a feature extractor, while TopoART replaces the head, e.g., the classification head, which usually consists of a fully connected layer followed by a softmax layer. In this setting, the backbone of a pretrained deep neural network provides high-quality features. These features are fed into the TopoART network, which can be used as usual. For instance, it can be trained incrementally if new classes show up during operation. This combines the advantages of both neural network approaches. An implementation of TopoART-based layers for deep neural networks can be found here: TopoART layers.

3. Do all TopoART networks have two modules?

No, the number of modules can be any positive integer. Usually, one to three modules are sufficient.

4. Are TopoART networks very sensitive to noise?

No, TopoART has a powerful noise reduction mechanism controlled by the parameters φ and τ. As a rule of thumb, if the results are disturbed by noise, increase φ.

5. Does TopoART use Fuzzy ART modules?

Not really, although there are some similarities. If β_sbm = 0 and φ = 1, the categories learnt by a TopoART module are equivalent to those of a Fuzzy ART module using complement coding, fast learning (β = 1), and the same values for the vigilance parameter ρ and the choice parameter α. In contrast, Fuzzy ART cannot group categories into clusters.

6. What can I do if the results contain no noise categories but the expected clusters are split into several fragments?

Most likely, the value of φ is too high. If decreasing φ lets too much noise into the results, increasing τ might help.

7. Why does the clustering result contain isolated categories in addition to the expected clusters?

There are three main reasons why this effect might show up:

1. The dataset contains small hotspots which are represented by the categories in question. This could be caused by random variations in the training data.
2. The isolated categories lie in close proximity to a larger cluster. If so, they are likely to be connected to this cluster as learning continues.
3. The data distribution might have changed, and the isolated categories constitute the starting point of a new cluster.

Please keep in mind that TopoART is a lifelong learning neural network. Thus, every output is just a snapshot of the current network state. In particular, existing categories can grow and new categories or links may be added later on.

8. What are the requirements for the input data of Episodic TopoART?

The following holds for all TopoART networks, not only for Episodic TopoART. The individual input values should reflect the properties of the data that matter for the application at hand. If the input data consists of features extracted from individual images of a video stream, for instance, these features should change to a similar degree as the relevant image information. Both low-level and high-level features can be used.