Option Trading, Analysis Pricing ActiveX Software .NET Components, Principal Component Analysis

Quick Links

Products

ActiveX/COM Components

Support

About Us


Maths Active/COM Controls Software Suite

		MathsStudio/X 5.0
		Data Analysis Suite of ActiveX Controls

Would you like to use Independent Component Analysis in your Windows programs: Visual Basic, Visual C++, Borland C++ Builder, Excel, Access etc?

Maths Studio/X is a premier suite of tools, containing ICA/X, PCA/X, Matrix Tools/X, MLP/X, Input Selection/X and Classifier/X.

Maths Studio/X is ideally suited to comprehensive data analysis tasks. It contains six of our main data analysis components: ICA/X and PCA/X implement an independent component analysis algorithm and principal component analysis algorithm respectively. Matrix Tools/X provides a set of tried and tested matrix functions which enable the development of robust numerical algorithms in the language of your choice operating under the Windows platform. MLP/X is an implementation of the popular neural network models: the Multilayer Perceptron and the Time Delay Neural Network. In MLP/X, both standard first order backpropagation and the second order Kalman-based learning algorithm (MEKA) are available.

The problem of input variable selection is well known in the task of modeling real world data. In many real world modeling problems, for example in the context of biomedical, industrial, or environmental systems, a problem can occur when developing multivariate models and the best set of inputs to use are not known. Input Selection/X implements a method of selecting a set of input variables using higher order statistics. Classifier/X will let you quickly and easily implement a k-means classifier in your programs.

Each component is both an ActiveX Control and a COM object, so they can be used in a wide range of applications that support these standards, including Visual Basic, Visual C++ and Excel. The controls are written as lightweight ATL C/C++ objects, and do not require bulky MFC DLLs. Because the controls are written in ATL they are efficient and small in size. The numerical processing is written in C for speed, and integrated into the lightweight ATL/C++ framework.

Components included:

ICA/X ActiveX Control and COM Object

PCA/X ActiveX Control and COM Object

Matrix Tools/X ActiveX Control and COM Object

Classifier/X ActiveX Control and COM Object

MLP/X ActiveX Control and COM Object

Input Selection/X ActiveX Control and COM Object

Purchase Maths Studio/X subscription now and lock in free upgrades and new components for 12 months without paying any more.

ICA/X ActiveX and COM Component

ICA/X ActiveX Component drops into most windows application programs and with just a few lines of code you will be able to implement ICA. With full source samples you will be able to quickly and easily include independent component analysis functionality in your applications.

ICA/X ActiveX Control implements an independent component analysis algorithm. ICA/X is both an ActiveX Control and a COM object, so it can be used in a wide range of applications that support these standards. This includes Visual Basic, Visual C++, Excel, Delphi and Borland C++ Builder.

The control is written as a lightweight ATL C/C++ object, and does not require bulky MFC DLLs. Because the control is written in ATL it is efficient and small in size. The numerical processing is written in C for speed, and integrated into the lightweight ATL/C++ framework.

Screen shot of an application built in Visual Basic using ICA/X.

Independent Component Analysis

Independent component analysis can be considered an extension of principal component analysis (PCA). In PCA, the input data is decorrelated to find the componentsthat are maximally correlated according to second order statistics. PCA gives orthogonalized and normalized outputs according to the second order statistics by minimizing the second order moments. The principal components can still be dependent however. In ICA, the aim is to process a number of measured signal vectors X and extract a set of statistically independent vectors Y which are estimates of some unknown source signals S which have been linearly mixed together via a mixing matrix A to form the observed input data. ICA seeks to ensure maximum independence, typically by minimizing the higher order moments of the outputs. When the higher order moments are zero (for non-gaussian input signals), the outputs are independent.

The problem of independent component analysis or blind source separation of signals mixed instantaneously, is defined as follows. We have available to us, a multivariate time series {x_i(t)}, i = 1,...,n. It is assumed that these signals are the result of a mixing process defined by

x_i(t) =

n
å
j = 1

a_ijs_j(t) .

We seek estimate the unknown signals s_j (t) using only the observed data vector x(t) = [x₁(t),x₂(t),...,x_n(t)]^T . The problem is to find a demixing matrix B such that y(t) = Bx(t)= BAs(t), where A is the unknown mixing matrix. This matrix mixes signals instantaneously in time. It is possible for there to be differing numbers of sensors than sources, that is, A may not be square. If the number of sources is the same as the number of sensors or observed inputs n, then A is a square n×n matrix. If B = A^-1, then y(t) = s(t), and perfect separation occurs. In practice, the optimum y will be some permutated and scaled version of s , since it is only possible to find B such that BA = PD where P is a permutation matrix and D is a diagonal scaling matrix (Tong, et. al. 1991).

ICA is useful in a wide range of data analysis tasks where we seek to understand and analyse sequential or time-based data in terms of their underlying composition, where the composition signals are as independent of each other as possible.

Many algorithms have been proposed to perform ICA. These may be divided into block-based or on-line adaptive techniques. Block-based algorithms take all the data in at once and produce the output. On-line adaptive algorithms process each data point in a continuous sense. A disadvantage of many algorithms, especially on-line adaptive algorithms, is the need to select tuning parameters such as the learning rate. If the learning rate is chosen to be too small, then the solution may not be found or it may be found very slowly. If the learning rate is chosen too large, then the algorithm may 'blow-up'. For many practical problems, it is useful to be able to have an algorithm that is capable of finding a solution without user intervention at all.

ICA/X implements the JADE algorithm (Cardoso, 1993) and is capable of separating linearly mixed input signals in a block-based manner without user intervention. The quality of the solution can vary depending on the specific characteristics of the data.

PCA/X ActiveX and COM Component

PCA/X ActiveX Control implements a principal component analysis algorithm. PCA/X is both an ActiveX Control and a COM object, so it can be used in a wide range of applications that support these standards. This includes Visual Basic, Visual C++, Excel, Delphi and Borland C++ Builder.

PCA/X ActiveX Control is written as a lightweight ATL C/C++ object for speed and is small in size. It weighs in at under 170kB and does not require bulky MFC DLLs.

Screen shot of an application built in Visual Basic using PCA/X.

Background of PCA

Principal component analysis (PCA) is a statistical data analysis technique used to reduce the dimensionality of multivariate data. Understanding multivariate data can be difficult due to the number of dimensions. When more than three dimensions are being analyzed, it is difficult to visualize the data.
PCA is a well known method for reducing the dimensionality by extracting components which are uncorrelated with each other. Moreover, the first component explains as much of the variance as possible, while remaining uncorrelated with subsequent principal components. Hence it is the first principal component. Further principal components in turn, provide as much of the remaining variance as possible.

PCA is useful in a wide range of data analysis tasks where we seek to understand and analyse sequential or time-based data in terms of their underlying composition, where the composition signals are as independent of each other as possible.

PCA/X employs singular value decomposition to separate linearly mixed input signals in a block-based manner without user intervention.

With full source code samples you can download and use immediately, Classifier/X will let you quickly and easily implement a k-means classifier in your programs. Download Classifier/X now and you can be developing programs immediately.

Classifier/X ActiveX and COM Component

Classifier/X is both an ActiveX Control and a COM object that implements the k-means algorithm to classify multivariate input data. It can be used in a wide range of applications including Visual Basic, Visual C++, Excel, Delphi and Borland C++ Builder.

The control is written as a lightweight ATL C/C++ object, and does not require bulky MFC DLLs. Because the control is written in ATL it is efficient and small in size (under 170k in size!). The numerical processing is written in C for speed, and integrated into the lightweight ATL/C++ framework.

k-Means Classification

There exist many problems where we have data that needs to be classified into distinct groups, but we may not possess precise class descriptions or decision boundaries.

Screen shot of an application built in Visual Basic using Classifier/X.

In such cases, it can be appropriate to use unsupervised classification methods. One widely known unsupervised classification algorithm that is based on clustering the data into local regions, is the k-means algorithm.

The algorithm for k-means classification is a widely known algorithm for classification which is capable of providing useful performance, although it does have some limitations. We have a multivariate input data set X which is defined as an M x N matrix. There are M input points in N-dimensional space. It is assumed there exist k compact classes of data, where k < n. The data is classified by allocating each data point to a class and then iteratively moving the data points between classes until we obtain the tightest overall cluster of points in each class. The specific algorithm is defined as follows:

Choose the number of classes k.
If not supplied, randomly determine a set of k class centers from the data.
Classify each data point into the nearest class.
Compute the sample mean of each cluster.
Reclassify each data point to the cluster with the nearest mean.
If the change in the mean is small enough, stop. Otherwise go to step 4.

The k-means algorithm has several potential problems including:

The classifications depend on the initial values of the class centers chosen. This means suboptimal classifications may be found, requiring multiple runs with different initial conditions.
The selection of a spurious data point as a center may lead to no data points in the class, with the outcome that the center cannot be updated.
The classification results depend on the distance metric used. Various preprocessing techniques can be introduced to produce desired results.
The classification depends on the number of clusters selected.

The algorithm implemented in Classifier/X uses a plain vanilla version of the k-means algorithm and does not introduce any measures to avoid the above problems. In general, it is left to the user to implement any special data preprocesing, initial center selection and so on.

InputSelection/X ActiveX and COM Component

This is particularly true when using neural networks. In this case, unrequired inputs can significantly increase learning complexity. Input variable selection (IVS) is aimed at determining which input variables are required for a model.

Screen shot of an application built in Visual Basic using Input Selection/X.

The task is to determine a set of inputs which will lead to an optimal model in some sense. Problems which can occur due to poor selection of inputs include the following:

As the input dimensionality increases, the computational complexity and memory requirements of the model increase.
Learning is more difficult with unrequired inputs.
Misconvergence and poor model accuracy may result from additional unrequired inputs.
Understanding complex models is more difficult than simple models which give comparable results.

This component implements an input variable selection algorithm using higher order cross statistics for each of the individual variables. In this component, the relevant inputs are determined directly, without using independent component analysis as documented in [1]. It is also possible to combine both components to select a potentially smaller number of relevant inputs for modeling.

The input variable selection used in this component is based on performing a statistical test between each of the input variable(s) and the desired output from the model. In some situations there may be dependence between input variables which leads to an overestimation of the number of inputs required. One method to overcome this is to use independent component analysis (ICA) as a preprocessing method. Input Selection/X is based on the method described in [1], but does not use ICA, and is different in several respects. Note that it is possible to use ICA/X as a preprocessor.

In order to assess the dependence between inputs and the desired system output, we use a method based on higher order cross moments, up to a specified order among the individual terms, and normalized in such a manner as to allow their direct comparison. This statistical measure can be used to establish the independence or otherwise of non-Gaussian signals. These cross moments are defined between the inputs x₁,x₂,...,x_n, individually at time t, and the target output y, with powers up to p=3. Not all cross terms are used, but a selection. The model implements only instantaneous moments, without employing time delays, however it is possible to use lagged regression vectors as inputs to achieve this result. The resulting output is a score vector indicating the dependence of each input on the output. This vector is then classified into to classes using the k-means algorithm to give a binary classification vector. In many instances it may be desirable to consider the score vector results, since the k-means algorithm will not always indicate the most appropriate inputs to consider. For example, it may leave out some inputs, but inspection of the score vector will display inputs which may also be reasonably considered as inputs to the model. Thus, it is possible to use human judgement for the classification of inputs as relevant.

Because the algorithm uses higher order statistics, it is capable of finding inputs in nongaussian and nonlinear processes. At the present time however, we do not implement higher order cross moment terms between inputs. This ensures the computational requirements are relatively low, while providing a reasonable chance of determining the inputs required. Note however, that depending on the statistical nature of the system being considered and the statistical relationships observed in the data, complex functions of inputs may not necessarily be determined. Thus, caution needs to be exercised. It is recommended that periodic testing be carried out to determine the effectiveness of the inputs being used. Also, spurious correlations or dependencies may exist between unrelated variables and hence could lead to falsely included inputs, eg: generated by coupled systems.

MatrixTools/X ActiveX and COM Component

Matrix Tools/X ActiveX Control implements a range of useful matrix functions including SVD, Sorting, Inverse, Eigenvalue decomposition, Covariance, Correlation and more. Matrix Tools/X is both an ActiveX Control and a COM object, so it can be used in a wide range of applications that support these standards. This includes Visual Basic, Visual C++, Excel and Borland C++ Builder. The control is written as a lightweight ATL C/C++ object, and does not require bulky MFC DLLs. Because the control is written in ATL it is efficient and small in size. The numerical processing is written in C for speed, and integrated into the lightweight ATL/C++ framework.

Matrix Tools/X provides a set of tried and tested matrix functions which enable the development of robust numerical algorithms in the language of your choice operating under the Windows platform. Because it is written in C and implemented as an ActiveX Control and COM object, you do not have to worry about how or if the library will be compatible with your code.

Matrix Tools/X is both an ActiveX Control and a COM object, implemented in a single DLL, yet it can be used in a wide range of programming languages that support these standards. This includes Visual Basic 6, Visual Basic .NET, Visual C++, Visual C#, Excel - VBA, Delphi and Borland C++ Builder.

Matrix Tools/X Platinum features ASP capability and is supplied with a special ASP-capable runtime DLL that can be used to do mathematical calculations in an ASP web site! The functions in MatrixTools/X and Matrix Tools/X Platinum are identical except for the ASP capabilities.

This is a sample of the functions included:

Cos, Sin, Tan, ACos, ASin, ATan,
Min, Max,
Sort
Mean, StdDev, Median
Ceil,
Cov, Variance, XCorr, XCov
Log, Log10
LoadData, SaveData
EIG, LU, SVD, SVDSolve
MaxPoly, MinPoly
Transpose
TrimMean
Rand
Det

Source Code Examples

Matrix Tools/X now includes sample applications - with full source code, in the following programming languages:

          Visual C#
          Visual C++.NET
          Visual Basic .NET
          Delphi
          Visual Basic 6
          Excel

Do you need to perform matrix computations in a different programming environment? Let us know and we will try to help.

Matrix Functions

Curve fitting sample program created using Matrix Tools/X.
Other sample applications with full source code are also included.

If you do have any questions about using our controls just email us and we will assist you in learning how to use our controls. Just take a look at our testimonials page to see some of the comments we have received from customers.

If you need quick results with industrial-strength reliability when using a neural network in your application, whether it is in Visual Basic, Visual C#, Visual C++, Delphi or Borland C++ Builder, then try MLP/X Multilayer Perceptron Active Control and COM Object.

MLP/X is an ActiveX control that is tightly coded in C and C++, and can be used in a wide range of applications. You can now implement neural network software in just minutes by dropping in our control.

Example Neural Networks with Free Source Code

MLP/X lets you quickly and easily implement a Multilayer Perceptron or Time Delay Neural Network in your application. With full source samples in Visual Basic, Excel Visual C++ coming soon), you will be able to quickly and easily implement a neural network in your application. We are serious about ensuring you can use our neural network software in your application. If you aren't sure about how to use an ActiveX control in your application, and need to have some help to get started, just send us an email and we will assist you.

Background on Neural Networks

Neural networks are a relatively recent, but widely used class of mathematical model which can be applied to problems such as time series prediction, classification, and function or functional approximation. In contrast to classical linear models, MLPs are used primarily because of their nonlinear modeling capabilities.

The sample application included shows how to use an MLP for solving function approximation, classification and time series prediction problems.

The Pima Indians Diabetes Classification task is performed using the K-Fold Cross Validation method to determine the best network size within a range.

Screen shot of an application built in Visual Basic using MLP/X.

MLP/X Learning Algorithms

MLP/X implements a two weight layer multilayer perceptron or a Time-Delay Neural Network (TDNN). The parameters for an MLP can be found using a range of different techniques. The most well known of these is the backpropagation algorithm which is implemented in MLP/X.

In addition to the backpropagation algorithm, an Extended Kalman Filter-based, second order algorithm is also implemented. This type of algorithm can provide faster convergence to a problem solution.

The particular EKF algorithm used is the MEKA algorithm [1]. Often the MEKA algorithm will provide better results than standard backpropagation, however this is dependent on the actual problem. Both methods use gradient descent and are therefore subject to becoming stuck in a local minima in the weight space. These algorithms belong to a class of neural networks learning algorithms termed "supervised learning". This means, that they are capable of learning from a given set of data which must be provided by the user.

MLP/X provides a range of simple functions to create, delete, load and save an MLP. The K-Fold Cross Validation method creates, trains and selects the optimal network size of an MLP in one step. The size is estimated in terms of the number of hidden units within a nominated range necessary to achieve minimal cross validation error. Network weights can be optionally saved for later recall. Cross validation errors are returned as well as the optimal number of hidden units

Technical Information

The ActiveX DLLs can be used in wide range of Windows applications. They requires no user interface and can be accessed by any ActiveX compatible development environment, including VB 6, Visual Basic .NET, Excel, VBA, Visual C#, Delphi and Visual C++. Each ActiveX and COM runtime component is included in a single DLL, these are typically about 200-300k in size.

The components supports threaded blocking and non-blocking modes. This means for lengthy computations, you can use the control in a program, pass it some data for processing and the program can then run other tasks and respond to user input while the computations are taking place. When processing is complete, an event is fired and the program continues from the data processing step. This blocking/non-blocking mode is under program control. Error codes are returned from the event indicating the success or otherwise of the data processing. The computations can also be interrupted under program control by the user, for example, it is straight forward to implement a "Stop" button to direct the computations to be stopped.

Matrix data used with the components and returned from the control can have different index starting values in Visual Basic. For example, you can choose to index your data from 0 or 1. The components pass the data back in an array indexed from the value you specify in a property of the control. All data used and returned is in double format. This means it is suitable for use with Visual Basic and Visual C++. In addition, all our controls are designed to function together, so you can build your applications quickly and easily.

References

A.D. Back and T.P. Trappenberg, "Selecting inputs for modelling using normalized higher order statistics and independent component analysis", IEEE Trans. on Neural Networks, Vol. 12, No. 3, pp. 612-617, May, 2001. https://andrewback.com
S. Shah and F. Palmieri, "MEKA - A fast, local algorithm for training feedforward neural networks", Intern. Joint Conf. Neural Networks (IJCNN), Vol. 3, 1990, pp. 41-46.
Jean-Francois Cardoso. Blind Signal Separation: Statistical Principles. Proc. IEEE, Special issue on blind identification and estimation, pp. 2009-2025, Vol 86, No. 10, 1998.
S. Amari and A. Cichocki. Blind signal processing - neural network approaches. Proc. IEEE, Special issue on blind identification and estimation, pp. 2026-2048, Vol 86, No. 10, 1998.
S. Amari, A. Cichocki, and H.H. Yang. A new learning algorithm for blind signal separation. In G. Tesauro, D.S. Touretzky, and T.K. Leen, editors, Advances in Neural Information Processing Systems 8 (NIPS*95), pages 757-763, Cambridge, MA, 1996. The MIT Press.
J.F. Cardoso and A. Souloumiac. Blind beamforming for non-Gaussian signals. IEE Proc. F., 140(6):771-774, December 1993.
A. Cichocki and L. Moszczynski. New learning algorithm for blind separation of sources. Electronics Letters, 28(21):1986-1987, October 8 1992.
P. Comon. Independent component analysis - a new concept? Signal Processing, 36(3):287-314, 1994.
C. Jutten and J. Herault. Blind separation of sources, Part I: An adaptive algorithm based on neuromimetic architecture. Signal Processing, 24:1-10, 1991.
E. Oja. Neural networks, principal components and subspaces. International Journal of Neural Systems, 1:61-68, 1989.
L. Tong, R.W. Liu, V.C. Soon, and Y.F. Huang. Indeterminacy and identifiability of blind identification. IEEE Trans. Circuits, Syst., 38(5):499-509, May 1991.