doc. Ing. Pavel Kordík, Ph.D.

Stať ve sborníku

Katedra aplikované matematiky

The evaluation of recommendation systems is a complex task. The offline and online evaluation metrics for recommender systems are ambiguous in their true objectives. The majority of recently published papers benchmark their methods using ill-posed offline evaluation methodology that often fails to predict true online performance. Because of this, the impact that academic research has on the industry is reduced. The aim of our research is to investigate and compare the online performance of offline evaluation metrics. We show that penalizing popular items and considering the time of transactions during the evaluation significantly improves our ability to choose the best recommendation model for a live recommender system. Our results, averaged over five large-size real-world live data procured from recommenders, aim to help the academic community to understand better offline evaluation and optimization criteria that are more relevant for real applications of recommender systems.

Článek

10.1007/s41060-023-00418-4

Katedra softwarového inženýrství
Katedra aplikované matematiky

Every real-life environments where users interact with items (products, films, research expert profiles) have several development phases. In the Cold-start phase, there are almost no interactions among users and items content-based recommendation systems (RS) can only recommend based on matching the attributes of the items. In the transition state, items start to collect user interactions but still a significant number of items have too small number of interactions, RS does not allow users to discover cold items. In a regular state, where most of the items in the system have enough interactions, the recommendations often suffer from low diversity of the items within a single recommendation. This article proposes a general recommendation algorithm based on Ontological-similarity, which is designed to address all the above problems. Our experiments show that recommendations generated by our approach are consistently better in all environment development phases and increase the success rate of recommendations by almost 50% measured using ontology-aware recall, which is also introduced in this article.

Článek

10.1007/978-3-031-42941-5_52

Katedra softwarového inženýrství
Katedra aplikované matematiky

Many recommendation systems struggle with the cold-start problem, especially in the early stages of a new application, when there are few active users and limited interactions. Traditional approaches like Collaborative Filtering cannot provide enough recommendations, and text-based methods, while helpful, do not offer sufficient context. This paper argues against the idea that the cold-start phase will eventually disappear and proposes a solution to enhance recommendation performance from the start. We propose using Ontologies and Knowledge Graphs to add a semantic layer to text-based methods and improve the recommendation performance in cold-start scenarios. Our approach uses ontologies to generate a knowledge graph that captures item text attributes’ implicit and explicit characteristics, extending the item profile with similar semantic keywords. We evaluate our method against state-of-the-art text feature extraction techniques and present the results of our experiments.

Článek

10.1007/978-3-031-42519-6_21

Katedra softwarového inženýrství
Katedra aplikované matematiky

Modern technical universities help students get practical experience. They educate thousands of students and it is hard for them to connect individual students with relevant industry experts and opportunities. This article aims to solve this problem by designing a matchmaking procedure powered by a recommendation system, an ontology, and knowledge graphs. We suggest improving recommendations and reducing the cold-start problem with a re-ranking module based on student educational profiles for students who opt-in. Each student profile is represented as a knowledge graph derived from the successfully completed courses of the individual. The system was tested in an online experiment and demonstrated that recommendations based on student educational profiles and their interaction history significantly improve conversion rates over non-personalised offers.

Stať ve sborníku

10.1109/CSIT56902.2022.10000471

Katedra aplikované matematiky

We introduce dynamic auto-sizing, a novel approach to training artificial neural networks which allows the models to automatically adapt their size to the problem domain. The size of the models can be further controlled during the learning process by modifying the applied strength of regularization. The ability of dynamic auto-sizing models to expand or shrink their hidden layers is achieved by periodically growing and pruning entire units such as neurons or filters. For this purpose, we introduce weighted L1 regularization, a novel regularization method for inducing structured sparsity. Besides analyzing the behavior of dynamic auto-sizing, we evaluate predictive performance of models trained using the method and show that such models can provide a predictive advantage over traditional approaches.

Stať ve sborníku

Katedra aplikované matematiky

Online learning is a well-established problem in machine learning. But while online learning is commonly concerned with learning on a stream of samples, this article is concerned with learning on a stream on features. An online quadratic discriminant analysis (QDA) is proposed because it is fast, capable of modeling feature interactions, and it can still return an exact solution. When a new feature is inserted into a training set, the proposed implementation of QDA showed a 1000-fold speed up to scikit-learn QDA. Fast learning on a stream of features provides a data scientist with timely feedback about the importance of new features during the feature engineering phase. In the production phase, it reduces the cost of updating a model when a new source of potentially useful features appears.

Stať ve sborníku

10.1109/IJCNN55064.2022.9892364

Katedra aplikované matematiky

Designing faster optimization algorithms is of ever-growing interest. In recent years, learning to learn methods that learn how to optimize demonstrated very encouraging results. Current approaches usually do not effectively include the dynamics of the optimization process during training. They either omit it entirely or only implicitly assume the dynamics of an isolated parameter. In this paper, we show how to utilize the dynamic mode decomposition method for extracting informative features about optimization dynamics. By employing those features, we show that our learned optimizer generalizes much better to unseen optimization problems in short. The improved generalization is illustrated on multiple tasks where training the optimizer on one neural network generalizes to different architectures and distinct datasets.

Stať ve sborníku

10.1109/CSIT56902.2022.10000782

Katedra softwarového inženýrství
Katedra aplikované matematiky
Fakulta informačních technologií

Offline optimization of recommender systems is a difficult task. Popular optimization criteria such as RMSE, Recall, and NDCG do not correlate much with online performance, especially when the recommendation algorithm is largely different from the one used to generate the offline data. An exciting direction of research to mitigate this problem is to use more robust optimization criteria. Serendipity is reported to be a promising proxy. However, more variants exist, and it is unclear whether they can be used as a single criterion to optimize. This paper analyzes how serendipity relates to other optimization criteria for three different recommendation algorithms. Based on our findings, we propose to modify the way serendipity is computed. We conduct experiments using three collaborative filtering algorithms: K-Nearest Neighbors, Matrix Factorization, and Embarrassingly Shallow Autoencoder (EASE). We also employ and evaluate the ensemble learning approach and analyze the importance of the individual components of serendipity.

Stať ve sborníku

10.1145/3523227.3551469

Katedra aplikované matematiky

Making recommender systems more transparent and auditable is crucial for the future adoption of these systems. Available tools typically present mostly errors of models aggregated over all test users, which is often insufficient to uncover hidden biases and problems. Moreover, the emphasis is primarily on the accuracy of recommendations but less on other important metrics, such as the diversity of recommended items, the extent of catalog coverage, or the opportunity to discover novel items at bestsellers’ expense. In this work, we propose RepSys, a framework for evaluating recommender systems. Our work offers a set of highly interactive approaches for investigating various scenario recommendations, analyzing a dataset, and evaluating distributions of various metrics that combine visualization techniques with existing offline evaluation methods. RepSys framework is available under an open-source license to other researchers.

Stať ve sborníku

10.1145/3523227.3551482

Katedra aplikované matematiky

Recently, the RS research community has witnessed a surge in popularity for shallow autoencoder-based CF methods. Due to its straightforward implementation and high accuracy on item retrieval metrics, EASE is potentially the most prominent of these models. Despite its accuracy and simplicity, EASE cannot be employed in some real-world recommender system applications due to its inability to scale to huge interaction matrices. In this paper, we proposed ELSA, a scalable shallow autoencoder method for implicit feedback recommenders. ELSA is a scalable autoencoder in which the hidden layer is factorizable into a low-rank plus sparse structure, thereby drastically lowering memory consumption and computation time. We conducted a comprehensive offline experimental section that combined synthetic and several real-world datasets. We also validated our strategy in an online setting by comparing ELSA to baselines in a live recommender system using an A/B test. Experiments demonstrate that ELSA is scalable and has competitive performance. Finally, we demonstrate the explainability of ELSA by illustrating the recovered latent space.

Stať ve sborníku

Katedra aplikované matematiky

Detrending and deseasoning is a common preprocessing step in time-series analysis. We argue that the same preprocessing step should be considered on relational data whenever the observations are time-dependent. We applied Hierarchical Generalized Additive Models (HGAMs) to detrend and deseason (D&D) 18 real-world relational datasets. The observed positive effect of D&D on the predictive accuracy is statistically significant. The proposed method of D&D might be used to improve the predictive accuracy of churn, default, or propensity models, among others.

Stať ve sborníku

10.1109/ICTAI52525.2021.00098

Katedra aplikované matematiky

In this article, we define an approximate generalization of aggregate functions for relational data with temporal attributes. This generalization is parametrized to allow simulation of a range of common aggregate functions and optionally take into account time. The parameters are not optimized, but we rather rely on repeated stochastic sampling of the parameters. We then apply a common regularized linear model to train a model on this high-dimensional space. Experimental results on 11 datasets suggest that there are datasets where incorporating time dimension into the model leads to an improvement in the predictive accuracy of the trained models.

Stať ve sborníku

10.1007/978-3-030-86383-8_11

Katedra aplikované matematiky

The recently introduced Embarrasingelly Shallow Autoencoder (EASE) algorithm presents a simple and elegant way to solve the top-N recommendation task. In this paper, we introduce Neural EASE to further improve the performance of this algorithm by incorporating techniques for training modern neural networks. Also, there is a growing interest in the recsys community to utilize variational autoencoders (VAE) for this task. We introduce Focal Loss Variational AutoEncoder (FLVAE), benefiting from multiple non-linear layers without an information bottleneck while not overfitting towards the identity. We show how to learn FLVAE in parallel with Neural EASE and achieve state-of-the-art performance on the MovieLens 20M dataset and competitive results on the Netflix Prize dataset.

Stať ve sborníku

10.1007/978-3-030-86340-1_19

Katedra aplikované matematiky

Small neural networks with a constrained number of trainable parameters, can be suitable resource-efficient candidates for many simple tasks, where now excessively large models are used. However, such models face several problems during the learning process, mainly due to the redundancy of the individual neurons, which results in sub-optimal accuracy or the need for additional training steps. Here, we explore the diversity of the neurons within the hidden layer during the learning process, and analyze how the diversity of the neurons affects predictions of the model. As following, we introduce several techniques to dynamically reinforce diversity between neurons during the training. These decorrelation techniques improve learning at early stages and occasionally help to overcome local minima faster. Additionally, we describe novel weight initialization method to obtain decorrelated, yet stochastic weight initialization for a fast and efficient neural network training. Decorrelated weight initialization in our case shows about 40% relative increase in test accuracy during the first 5 epochs.

Stať ve sborníku

Katedra teoretické informatiky
Katedra počítačových systémů
Katedra aplikované matematiky

The paper introduces a novel age-at-death estimation model based on Convolutional Neural Network (CNN). The model uses 3D scan of human pubic symphysis as an input and estimates the age-at-death of the individual as an output. The Mean Absolute Error (MAE) of this model is about 10.6 years for individuals between 18 and 92 years of age-at-death. Moreover, the results of the study indicate that pubic symphysis can be used to estimate the age of individuals across the entire age range. The study involved a sample of 483 bone scans collected from 374 individuals (from which 109 individuals provided both left and right pubic symphysis).

Stať ve sborníku

10.1109/ICTAI52525.2021.00012

Katedra aplikované matematiky

Stratified cross-validation is one of the standard methods of how to evaluate classifier’s generalization accuracy. However, conventional implementations of cross-validation can stratify only by a single column. In this paper, we propose to utilize Integer Linear Programming in order to enable stratification by multiple columns. Our experiments using an extensive set of multi-label data sets shows that the proposed method significantly outperforms non-stratified cross-validation.

Stať ve sborníku

Katedra aplikované matematiky

The MetaDL Challenge 2020 focused on image classification tasks in few-shot settings. This paper describes second best submission in the competition. Our meta learning approach modifies the distribution of classes in a latent space produced by a backbone network for each class in order to better follow the Gaussian distribution. After this operation which we call Latent Space Transform algorithm, centers of classes are further aligned in an iterative fashion of the Expectation Maximisation algorithm to utilize information in unlabeled data that are often provided on top of few labelled instances. For this task, we utilize optimal transport mapping using the Sinkhorn algorithm. Our experiments show that this approach outperforms previous works as well as other variants of the algorithm, using K-Nearest Neighbour algorithm, Gaussian Mixture Models, etc.

Článek

10.1145/3299876

Katedra aplikované matematiky
Fakulta informačních technologií

Traditional clustering algorithms fail to produce human-like results when confronted with data of variable density, complex distributions, or in the presence of noise. We propose an improved graph-based clustering algorithm called Chameleon 2, which overcomes several drawbacks of state-of-the-art clustering approaches. We modified the internal cluster quality measure and added an extra step to ensure algorithm robustness. Our results reveal a significant positive impact on the clustering quality measured by Normalized Mutual Information on 32 artificial datasets used in the clustering literature. This significant improvement is also confirmed on real-world datasets. The performance of clustering algorithms such as DBSCAN is extremely parameter sensitive, and exhaustive manual parameter tuning is necessary to obtain a meaningful result. All hierarchical clustering methods are very sensitive to cutoff selection, and a human expert is often required to find the true cutoff for each clustering result. We present an automated cutoff selection method that enables the Chameleon 2 algorithm to generate high-quality clustering in autonomous mode.

Stať ve sborníku

Katedra aplikované matematiky

We study an interesting and challenging problem, supervised learning on a stream of features, in which the size of the feature set is unknown, and not all features are available for learning while leaving the number of observations constant. In this problem, the features arrive one at a time, and the learner’s task is to train a model equivalent to a model trained from "scratch". When a new feature is inserted into the training set, a new set of trees is trained and added into the current forest. However, it is desirable to correct the selection bias: older features has more opportunities to get selected into trees than the new features. We combat the selection bias by adjusting the feature selection distribution. However, while this correction works well, it may require training of many new trees. In order to keep the count of the new trees small, we furthermore put more weight on more recent trees than on the old trees.

Stať ve sborníku

Katedra aplikované matematiky
Fakulta informačních technologií

Recommender systems are usually trained and evaluated on historical data. Offline evaluation is, however, tricky and offline performance can be an inaccurate predictor of the online performance measured in production due to several reasons. In this paper, we experiment with two offline evaluation strategies and show that even a reasonable and popular strategy can produce results that are not just biased, but also in direct conflict with the true performance obtained in the online evaluation. We investigate offline policy evaluation techniques adapted from reinforcement learning and explain why such techniques fail to produce an unbiased estimate of the online performance in the “watch next” scenario of a large-scale movie recommender system. Finally, we introduce a new evaluation technique based on Jaccard Index and show that it correlates with the online performance.

Článek

10.1007/s10994-017-5682-0

Katedra teoretické informatiky
Katedra aplikované matematiky

Recent meta-learning approaches are oriented towards algorithm selection, optimization or recommendation of existing algorithms. In this article we show how data-tailored algorithms can be constructed from building blocks on small data sub-samples. Building blocks, typically weak learners, are optimized and evolved into data-tailored hierarchical ensembles. Good-performing algorithms discovered by evolutionary algorithm can be reused on data sets of comparable complexity. Furthermore, these algorithms can be scaled up to model large data sets. We demonstrate how one particular template (simple ensemble of fast sigmoidal regression models) outperforms state-of-the-art approaches on the Airline data set. Evolved hierarchical ensembles can therefore be beneficial as algorithmic building blocks in meta-learning, including meta-learning at scale.

Stať ve sborníku

Katedra aplikované matematiky

Many feature selection methods were developed in the past, but in the core, they all work the same way — you pass a set of features to the algorithm and get a reduced set of the features. But can we perform a non-trivial feature selection without first observing the features? This is an important question because if we were actually able to predict feature importance before observing the features, we would reduce computation requirements of all stages of machine learning process beginning with feature engineering. In this article, we argue that it is possible to predict feature importance before feature vector observation. The trick is that we use meta-features about the features to perform the feature selection. We evaluate the concept on 15 relational databases. On average, it was enough to generate the top decile of all features to get the same model accuracy as if we generated all features and passed them to the model.

Stať ve sborníku

10.1007/978-3-319-70581-1_18

Katedra teoretické informatiky

Scalability of predictive models is often realized by data subsampling. The generalization performance of models is not the only criterion one should take into account in the algorithm selection stage. For many real world applications, predictive models have to be scalable and their training time should be in balance with their performance. For many tasks it is reasonable to save computational resources and select an algorithm with slightly lower performance and significantly lower training time. In this contribution we made extensive benchmarks of predictive algorithms scalability and examined how they are capable to trade accuracy for lower training time. We demonstrate how one particular template (simple ensemble of fast sigmoidal regression models) outperforms state-of-the-art approaches on the Airline data set.

Stať ve sborníku

Katedra aplikované matematiky

One of the most common methods for classifier comparison is Friedman’s test. However, Friedman’s test has a known flaw — ranking of classifiers A and B does not depend only on the properties of classifiers A and B, but also on the properties of all other evaluated classifiers. We illustrate the issue on a question: “What is better, bagging or boosting?”. With Friedman’s test, the answer depends on the presence/absence of irrelevant classifiers in the experiment. Based on the application of Friedman’s test on an experiment with 179 classifiers and 121 datasets we conclude that it is very easy to game the ranking of two insignificantly different classifiers. But once the difference becomes significant, it is unlikely that by removing irrelevant classifiers we obtain significant results with reversed ranking.

Stať ve sborníku

Katedra teoretické informatiky

For relational learning, it is important to know the relationships between the tables. In relational databases, the relationships can be described with foreign key constraints. However, the foreign keys may not be explicitly specified. In this article, we present how to automatically and quickly identify primary & foreign key constraints from metadata about the data. Our method was evaluated on 72 databases and has F-measure of 0.87 for foreign key constraint identification. The proposed method significantly outperforms in runtime related methods reported in the literature and is database vendor agnostic.

Stať ve sborníku

10.1109/STC-CSIT.2017.8098848

Katedra softwarového inženýrství
Katedra aplikované matematiky

Recent meta-learning approaches are oriented towards algorithm selection, optimization or recommendation of existing algorithms. In this paper we show how inductive algorithms constructed from building blocks on small data subsample can be scaled up to model large data sets. We demonstrate how one particular template (simple ensemble of fast sigmoidal regression models) outperforms state-of-the-art approaches on the Airline data set.

Stať ve sborníku

10.1109/ICDMW.2016.0123

Katedra teoretické informatiky

Many clustering evaluation methods are computed as a ratio between two objectives, typically these objectives express the compactness of all clusters while trying to maximize the separation between individual clusters. However, the clustering process itself is typically implemented as a single objective problem: optimizing a linear combination of between-points closeness. We propose MoCham - a hierarchical clustering algorithm that uses a multi-objective optimization for finding the optimal data points to merge. Our results suggest that a careful candidate selection of Pareto dominating pairs leads to more robust clustering results.

Stať ve sborníku

10.1109/IJCNN.2016.7727551

Katedra teoretické informatiky

The capacity of recurrent neural networks to learn complex sequential patterns is improving. Recent developments such as Clockwork RNN, Stack RNN, Memory networks and Neural Turing Machine all aim to increase long-term memory capacity of recurrent neural networks. In this study, we investigate properties of Neural Turing Machine, compare it with ensembles of Stack RNN on artificial benchmarks and applied it to learn human mobility patterns. We show, that Neural Turing Machine based predictor outperformed not only n-gram based prediction, but also neighborhood based predictor, that was designed to solve this particular problem. Our models will be deployed in anti-drug police department to predict mobility of suspects.

Stať ve sborníku

10.1109/IJCNN.2016.7727600

Katedra teoretické informatiky
Katedra softwarového inženýrství

Educational data can help us to personalise university information systems. In this paper, we show how educational data can be used to improve the performance of interaction-based recommender systems. Educational data is transformed to student profiles helping to prevent cold start problems when recommending projects to students with few user interactions. Our results show that our hybrid interaction based recommender boosted by educational profiles significantly outperforms bestseller recommendation, which is a mainstream recommendation method for cold start users.

Stať ve sborníku

10.1007/978-3-319-19644-2_39

Katedra teoretické informatiky

One of the biggest challenges in clustering is finding a robust and versatile criterion to evaluate the quality of clustering results. In this paper, we investigate the extent to which unsupervised criteria can be used to obtain clusters highly correlated to external labels. We show that the usefulness of these criteria is data-dependent and for most data sets multiple criteria are required in order to identify the best performing clustering algorithm. We present a multi-objective evolutionary clustering algorithm capable of finding a set of high-quality solutions. For the real world data sets examined the Pareto front can offer better clusterings than simply optimizing a single unsupervised criterion.

Stať ve sborníku

10.1007/978-3-319-19713-5_54

Katedra teoretické informatiky

We are focusing on an issue regarding how to actually recognize the skills of students based on educational results. Existing approaches do not offer suitable solutions. This paper will introduce algorithms making possible to aggregate educational results using ontology. We map the aggregated results, using various methods, as skills that are understandable for external partners and usable to recommend students for projects and projects for students. We compare the results of individual algorithms with subjective assessments of students, and we apply a recommendation algorithm that closely models these skills.

Stať ve sborníku

Katedra teoretické informatiky

In this paper we propose a clustering process which uses a multi-objective evolution to select a set of diverse clusterings. The selected clusterings are then combined using a consensus method. This approach is compared to a clustering process where no selection is applied. We show that careful selection of input ensemble members can improve the overall quality of the final clustering. Our algorithm provides more stable clustering results and in many cases overcomes the limitations of base algorithms.

Stať ve sborníku

10.1109/CIEL.2014.7015740

Katedra teoretické informatiky

The model selection stage is one of the most difficult in predictive modeling. To select a model with a highest generalization performance involves benchmarking huge number of candidate models or algorithms. Often, a final model is selected without considering potentially high quality candidates just because there are too many possibilities. Improper benchmarking methodology often leads to biased estimates of model generalization performance. Automation of the model selection stage is possible, however the computational complexity is huge especially when ensembles of models and optimization of input features should be also considered. In this paper we show, how to automate model selection process in a way that allows to search for complex hierarchies of ensemble models while maintaining computational tractability. We introduce two-stage learning, meta-learning templates optimized by evolutionary programming with anytime properties to be able to deliver and maintain data-tailored algorithms and models in a reasonable time without human interaction. Co-evolution if inputs together with optimization of templates enabled to solve algorithm selection problem efficiently for variety of datasets.

Stať ve sborníku

Katedra teoretické informatiky

Universities are seldom using their data efficiently. In this case study, we show how educational data can be used to recommend suitable students for project, get feedback from industrial partners, help students to focus on skills that are demanded by companies. We have developed portal for students collaboration with industrial partners and run it in a pilot for almost a year. Based on our observations described in this contribution, we are adjusting the portal to enhance the functionality and streamline processes supported by the portal.

Stať ve sborníku

10.1007/978-3-642-32922-7_3

Katedra teoretické informatiky

In the areas of Data Mining (DM) and Knowledge Discovery (KD), large variety of algorithms has been developed in the past decades, and the research is still ongoing. Data mining expertise is usually needed to deploy the algorithms available. Specifically, a process of interconnected actions referred to as knowledge flow (KF) needs to be assembled when the algorithms are to be applied to given data. In this paper, we propose an innovative evolutionary approach to automated KF synthesis and optimization. We demonstrate the evolutionary KF synthesis on the problem of classifier construction. Both preprocessing and machine learning actions are selected and configured by means of evolution to produce a model that fits very well for a given dataset.

Stať ve sborníku

10.1007/978-3-642-33018-6_48

Katedra teoretické informatiky

Clustering algorithms belong to a category of unsupervised learning methods which aim to discover underlying structure in a dataset without given labels. We carry out research of methods for an analysis of a biological time series signals, putting stress on global patterns found in samples. When clustering raw time series data, high dimensionality of input vectors, correlation of inputs, shift or scaling sensitivity often deteriorates the result. In this paper, we propose to represent time series signals by various parametric models. A significant parameters are determined by means of heuristic methods and selected parameters are used for clustering. We applied this method to the data of cell's impedance profiles. Clustering results are more stable, accurate and computationally less expensive than processing raw time series data.

Článek

10.1007/s13369-012-0385-0

Katedra teoretické informatiky

EL Fayoum-Cairo district lies in the north western part of Cairo city. It is affected by several earthquakes. According to the Egyptian Network Seismology of National Research Institute of Astronomy and Geophysics (NRIAG), the last one occurred in July 2005 with a magnitude of 4.2 in Richter scale. The Bouguer and the aeromagnetic anomaly data as well as the detailed land-magnetic survey have been subjected to different techniques of processing and interpretation to better understand the tectonic setting of the study area. For example, different kinds of filters such as 2D spectral analysis, 3D analytical signal, and Euler deconvolution techniques have been applied. Finally, the 2D modeling has been used to simulate the subsurface configuration along some selected profiles at the area. It can be noticed from the obtained results that, the seismic events (Dahshour earthquakes) are closely related to a major NNW normal fault which has a deep extension and slightly lateral displacement, in addition to its NE-conjugate faults. The recent seismic activities in the study area are directly related and/or associated with the rejuvenation of the lateral movements. Three-dimensional interpretations of the magnetic anomaly and Bouguer anomaly maps of Fayoum, Cairo province area, Northern Western Desert of Egypt have been presented. In addition, a high-resolution 3D magnetic and gravity model constrained with the seismic results reveals a possible crustal thickness and density distribution of the northern part of Egypt between the sedimentary cover and the mantel. The results reveal that the basement has an irregular surface with depths ranging from 3,600 to 5,500 m. ...

Článek

10.1016/j.simpat.2012.11.003

Katedra teoretické informatiky

Training data matrix used for classification of text documents to multiple categories is characterized by large number of dimensions while the number of manually classified training documents is relatively small. Thus the suitable dimensionality reduction techniques are required to be able to develop the classifier. The article describes two-step supervised feature extraction method that takes advantage of projections of terms into document and category spaces. We propose several enhancements that make the method more efficient and faster than it was presented in our former paper. We also introduce the adjustment score that enables to correct defected targets or helps to identify improper training examples that bias extracted features.

Stať ve sborníku

Katedra teoretické informatiky
Katedra počítačových systémů

Transforming of text documents to real vectors is an essential step for text mining tasks such as classification, clustering and information retrieval. The extracted vectors serve as inputs for data mining models. Large vocabularies of natural languages imply a high dimensionality of input vectors; hence a substantial dimensionality reduction has to be made. We propose a new approach to a vector representation of text documents. Our representation takes into account an order of latent topics that generate observed words; an extracted document vector includes information about the adjacency of words in a document. We experimentally proved that the proposed representation enables to build document classifiers of higher accuracy using shorter document vectors. Short but informative document vectors enable to save memory for storing data, to use simpler models that learn faster and to significantly reduce an overfit effect.

Stať ve sborníku

Katedra teoretické informatiky
Katedra počítačových systémů

Widely used document classifiers are developed over a bag-of-words representation of documents. Latent semantic analysis based on singular value decomposition is often employed to reduce the dimensionality of such representation. This approach overlooks word order in a text that can improve the quality of classifier. We propose language independent method that records the context of particular word into a context network utilizing products of latent semantic analysis. Words' contexts networks are combined to one network that represents a document. A new document is classified based on a similarity between its network and training documents networks. The experiments show that proposed classifier achieves better performance than common classifiers especially when a foregoing reduction of dimensionality is significant.

Stať ve sborníku

Katedra teoretické informatiky
Katedra počítačových systémů

The classification of text documents generally deals with large dimensional data. To favor generality of classification researcher has to apply a dimensionality reduction technique before building a classifier. We propose classification and reduction algorithm that makes use of latent uncorrelated topics extracted from training documents and their known categories. We suggest several latent feature selection options and provide their testing.

Stať ve sborníku

10.1109/CEC.2012.6256528

Katedra teoretické informatiky

We improved memory complexity of Max-Min Ant System algorithm, which is one of the best performing Ant Colony Optimization algorithms. Standard implementation uses n2 matrix of artificial pheromone to solve the Traveling Salesman Problem, where n is the number of cities. We show, that only small fraction of values in the pheromone matrix is changed from a global value and so the matrix can be replaced by an array of hash tables in order to reduce the memory complexity. This improvement is very useful in case of parallelization on multicore architectures, where frequent transfers of random parts of matrices cause radical slowdown. Also it enables the algorithm to be competitive with others when solving large instances.

Stať ve sborníku

10.1109/IJCNN.2012.6252379

Katedra teoretické informatiky

Meta-learning templates are data-tailored algo- rithms that produce supervised models. When a template is evolved on a particular dataset, it is supposed to generate good models not only on this data set but also on similar data. In this paper, we will investigate one possible way of measuring the similarity of datasets and whether it can be used to estimate if meta-learning templates produce good models. We performed experiments on several well known data sets from the UCI machine learning repository and analyzed both the similarity of datasets and templates in the space of performance meta- features (landmarking). Our results show that the most universal algorithms (in terms of average performance) for supervised learning are the complex hierarchical templates evolved by our SpecGen approach.

Kapitola v knize

10.5772/50787

Katedra teoretické informatiky

Many methods of Data Mining use data sets for setting their parameters, particularly training and testing sets. Setting of parameters corresponds to the learning (training) of the methods. It is e.g. a case of artificial neural networks and other adaptive (iterative) methods. Some of these methods utilize so-called validation set as well. A question that can arise is how to correctly divide or other way preprocess a given data set to these sets, i.e. how select data samples from the original set and place them into the training and testing sets. The chapter focuses on an overview of existing methods that deal with methods of data selection and sampling. A general approach to the problem of data selection to training, testing and eventually validation sets is discussed. To be able to compare individual approaches, model evaluation techniques are discussed as well. Data splitting is one of used approaches to construct training, testing and possibly validation sets, but there are many other approaches to proper data preparation. Instance selection methods and class balancing methods belong to these approaches and are discussed in the chapter. An overview of the methods together with their features, utilization, positives and negatives is given as well. Selected main principles are illustrated in figures.

Kapitola v knize

10.1007/978-3-642-20980-2_6

Katedra teoretické informatiky

The cornerstone of successful data mining is to choose a suitable modelling algorithm for given data. Recent results show that the best performance can be achieved by an efficient combination of models or classifiers. The increasing popularity of combination (ensembling, blending) of diverse models has been significantly influenced by its success in various data mining competitions.

Kapitola v knize

10.1007/978-3-642-20980-2_6

Katedra teoretické informatiky

The cornerstone of successful data mining is to choose a suitable modelling algorithm for given data. Recent results show that the best performance can be achieved by an efficient combination of models or classifiers. The increasing popularity of combination (ensembling, blending) of diverse models has been significantly influenced by its success in various data mining competitions.

Stať ve sborníku

Katedra teoretické informatiky

Multivariate data are difficult to explore. The most popular linear projection techniques mapping data to 2-dimensional space often fail to reveal the patterns of interest. Non-linear mapping techniques are both slow and inefficient. In this paper, we propose a heuristic that allows users to adjust the parameters of mapping techniques just by stating their preferences iteratively. The preliminary results on real-world dataset demonstrate the power of our approach.

Stať ve sborníku

Katedra teoretické informatiky

Recently, ensemble methods proved to improve accuracy of weak learners and reduce overfitting of models with high plasticity. In this paper, we experiment with various state of the art ensemble strategies applied to polynomial models. We also explore the efficiency of ensembling, when applied to polynomial models with increasing plasticity.

Stať ve sborníku

Katedra teoretické informatiky

In an age prediction problem is difficult to get good results for adults. Almost all people over twenty years of age have finalized the teeth mineralization phase. Therefore the age prediction for adults needs special methods for age determining. On the other hand, children have each tooth in one of several mineralization phases and the process of prediction is more precise. In this paper we present successful method for children age prediction on the basis of teeth mineralization (MFH method). For the age prediction we used a data mining software FAKE-GAME which uses for mining a special hybrid artificial neural network called GAME. This method provides two important approaches for the age prediction. It incorporates feature selection mechanism which selects only the most important teeth. It also derives feature ranking (selected teeth ranking) by using FeRaNGA method during a building phase of the GAME network - data mining model.

Stať ve sborníku

Katedra teoretické informatiky

Estimation of continuous parameters is frequent task in modelling and simulation. There are several general purpose algorithms available for this task. We benchmarked these algorithms in order to recommend an appropriate algorithm for our model identification problem. We present results of optimization algorithms for standard benchmarking functions and show the importance of proper parameter setting. When these algorithms are applied to the estimation of model parameters, results are quite different. For this task, the gradient (quasi-Newton) and the nature inspired method (CMAES) can be efficiently combined, achieving the best optimization performance.

Stať ve sborníku

Katedra teoretické informatiky

Almost all of Feature Ranking and Feature Selection methods are based on one of the specific approach only (e.g. statistical measure of the data, wrapper ap- proach, etc.). The problem is also in an applicability of these methods. Most of Feature Ranking and Feature Selection methods are designed for categorial data only. Some of them have also its special version for regression problems. In this paper we present modificated approach in methods for Feature Ranking and Fea- ture Selection based on a combination of statistical measure and utilization of data mining model. The main feature of this approach is that the data mining algorithm (GAME) is able to deal with numerical data as well as it can be applied to catego- rial data. It incorporates feature selection mechanism and methods, described in this paper, derive feature ranking from final data mining model.

Článek

10.1016/j.neunet.2010.02.003

Katedra teoretické informatiky

Optimization of neural network topology, weights and neuron transfer functions for given data set and problem is not an easy task. In this article, we focus primarily on building optimal feed-forward neural network classifier for i.i.d. data sets. We apply metalearning principles to the neural network structure and function optimization. We show that diversity promotion, ensembling, self-organization and induction are beneficial for the problem. We combine several different neuron types trained by various optimization algorithms to build a supervised feedforward neural network called Group of Adaptive Models Evolution (GAME). The approach was tested on wide number of benchmark data sets. The experiments show that the combination of different optimization algorithms in the network is the best choice when the performance is averaged over several real-world problems.

Stať ve sborníku

Katedra teoretické informatiky

Most of Feature Ranking and Feature Selection approaches can be used for categorial data only. Some of them rely on statistical measures of the data, some are tailored to a specific data mining algorithm (wrapper approach). In this paper we present new methods for feature ranking and selection obtained as a combination of the above mentioned approaches. The data mining algorithm (GAME) is designed for numerical data, but it can be applied to categorial data as well. It incorporates feature selection mechanisms and new methods, proposed in this paper, derive feature ranking from final data mining model. The rank of each feature selected by model is computed by processing correlations of outputs between neighboring model?s neurons in different ways. We used four different methods based on fuzzy logic, certainty factors and simple calculus. The performance of these four feature ranking methods was tested on artificial data sets, on well known Ionosphere data set and on well known Housing

Stať ve sborníku

Katedra teoretické informatiky

The data preprocessing is very important part of the knowledge discovery process. Data mining systems con- tains tens of preprocessing methods (for example methods for missing data imputation, data reduction, discretization, data enrichment, etc...) and usually it is not clear which methods to use. The selection of preprocessing methods appropriate for particular dataset needs strong experience and a lot of experimenting. In this paper we will test influence of modelling method which is the corner stone of Inductive Preprocessing Algorithm. Modelling method is used to evaluate evolved sequence of the preprocessing methods. In this paper we compare four modelling methods in respect to final achieved accuracy. The tested modelling methods are Polynomial model, Decision Tree, SVM and Logistic Function Classifier. To test our automatic preprocessing utilize several real-world datasets available from UCI Machine learning repository.

Stať ve sborníku

Katedra teoretické informatiky

An intelligent building automation system can manage many devices in order to balance the energy savings and comfort of the inhabitants. The strategy controlling devices has to be adaptive and learn to match users' needs. Based on the Amigo framework, we developed a simulator of a household. We added lights, devices, pheromone controlled inhabitants, physical model of heat loss, etc. The system runs real-time and a simple strategy was used to control heat, lights and other devices. In this simulator, we plan to evaluate several different control strategies in term of energy efficiency and user comfort. We also proposed an adaptive control strategy, based on the neural networks induced on data from sensors and user interaction signals. We built an experimental KNXbus platform demonstrating the feasibility of our concept.

Stať ve sborníku

10.1109/IEMBS.2009.5335161

Katedra teoretické informatiky

Complex fractionated atrial electrograms (CFAEs) may represent the electrophysiological substrate for atrial fibrillation (AF). Progress in signal processing algorithms to identify CFAEs sites is crucial for the development of AF ablation strategies. A novel algorithm for automated description of atrial electrograms (A-EGMs) fractionation based on wavelet transform and several statistical pattern recognition methods was proposed and new methodology of A-EGM processing was designed and tested. The algorithms for A-EGM classification were developed using normal density based classifiers, linear and high degree polynomial classifiers, nearest mean scaled classifiers, nonlinear classifiers, neural networks and j48. All classifiers were compared and tested using a representative set of 1.5 s A-EGMs (n = 68) ranked by 3 independent experts 100% coincidentialy into 4 classes of fractionation: 1 - organized atrial activity; 2 - mild; 3 - intermediate; 4 - high degree of fractionation. Feature extraction and well performing classification algorithms tested here showed maximal error of 15% and mean classification error across all implemented classifiers 9%, and the best mean classification error 5.9% (nearest mean classifier), and classification error of highly fractionated A-EGMs of approximately 9%.

Stať ve sborníku

Katedra teoretické informatiky

Most of Feature Ranking and Feature Selection approaches can be used for categorial data only. In this paper we present new methods for feature ranking and selection obtained as a combination of the above mentioned approaches. The data mining algorithm (GAME) is designed for numerical data, but it can be applied to categorial data as well. It incorporates feature selection mechanisms and new methods, proposed in this paper, derive feature ranking from final data mining model. The rank of each feature selected by model is computed by processing correlations of outputs between neighboring model's neurons in different ways. We used four different methods based on fuzzy logic, certainty factors and simple calculus. The performance of these four feature ranking methods was tested on artificial data sets and on well known real world data sets. These methods produce ranking consistent with recently published studies.

Kapitola v knize

Katedra teoretické informatiky

In this chapter, an algorithm to construct hybrid self-organizing neural network is proposed. It combines niching evolutionary strategies, nature inspired and gradient based optimization algorithms to evolve neural network with optimal topology adapted to a data set. The GAME algorithm is something in between the GMDH algorithm and the NEAT algorithm. It is capable to handle irrelevant inputs, short and noisy data samples, but also complex data such as "two intertwined spirals" problem. The selforganization of the topology allows it to produce accurate models for various tasks. Bencharking with machine learning algorithms implemented in the Weka software showed that the accuracy of GAME models was superior for both regression and classification problems. The most successful configuration of the GAME algorithm is not changing with problem character, natural evolution selects all important parameters of the algorithm. This is a significant step towards the automated data mini

Stať ve sborníku

Katedra teoretické informatiky

In this paper we present state of the art in the field of meta-optimization. We are looking for possible applications of Inductive Approach in this field. We propose an inductive based optimization strategy combining several optimization algorithms to solve increasingly complex problems.

Stať ve sborníku

Katedra teoretické informatiky

This work presents the modeling results of different real noisy data (nalada: humanities, spirals_1 and spirals_2: complex data, motol_brain: motol hospinal neurosurgery, boshouse: house prices and also artificial data), using soft computing methods - combinatorial group method of data handling (combi GMDH) and Group of adaptive models evolution (GAME) method. The goal of our work is to get the best possible result on such noisy data and to compare the results of particular methods. Also, we combine these two methods (Game_Combi_GMDH) to get better prediction.

Stať ve sborníku

Katedra teoretické informatiky

Purpose: Complex fractionated atrial electrograms (CFAEs) may represent the electrophysiologic substrate for atrial fibrillation (AF). Progress in signal processing algorithms and pattern recognition systems to identify CFAEs sites is crucial for the development of AF ablation strategies. Methods: Head-to-head comparison of two methods for quantification of atrial electrograms (A-EGMs) with different degree of fractionation was performed using representative set of 1.5s A-EGMs (n = 113) that were blindly ranked by 3 independent experts into 4 categories: 1 - organized atrial activity; 2 - mild; 3 - intermediate; 4 - high degree of fractionation. First method (M1) assessed the average interval between discrete A-EGM spikes using algorithm previously implemented in commercially available electroanatomic mapping system.

Stať ve sborníku

Katedra teoretické informatiky

The data preprocessing is very important part of the knowledge discovery process. Data mining systems contains tens of preprocessing methods (for example methods for missing data imputation, data reduction, discretization, data enrichment, etc...) and usually it is not clear which methods to use. The selection of preprocessing methods appropriate for particular dataset needs strong experience and a lot of experimenting. In this paper we will test our extension of inductive approach to data preprocessing. We developed inductive preprocessing method which utilizes genetic algorithm to compose from scratch a sequence of preprocessing methods which fits to the data and allows successful model to be created. To test our automatic preprocessing utilize several real-world datasets available from UCI Machine learning repository.