tag:blogger.com,1999:blog-75652592024-02-07T13:41:42.530+01:00Vince Vella's BlogMy musings, interesting links and online contributions.Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.comBlogger30125tag:blogger.com,1999:blog-7565259.post-61512173759156791822017-06-29T12:10:00.001+02:002017-06-29T12:15:42.012+02:00Simple moving average trading strategy using Python<br />
Hi All,<br />
<br />
I am presenting simple boiler point code that can quickly be applied to test technical indicator strategies using Python. The code:<br />
<br />
1. downloads daily stock data from google,<br />
2. calculates the short and long moving averages<br />
3. generates the trading signals<br />
4. calculates the daily returns<br />
5. runs the moving average strategy and calculates the cumulative return<br />
6. plots cumulative return of our simple strategy<br />
<br />
Here is the code ... enjoy trying it out and extend it as required:<br />
<br />
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Menlo'; font-size: 9.0pt;"><span style="color: #cc7832; font-weight: bold;">import </span>numpy <span style="color: #cc7832; font-weight: bold;">as </span>np
<span style="color: #cc7832; font-weight: bold;">import </span>pandas_datareader <span style="color: #cc7832; font-weight: bold;">as </span>datar
<span style="color: #cc7832; font-weight: bold;">import </span>datetime
<span style="color: #cc7832; font-weight: bold;">import </span>matplotlib.pyplot <span style="color: #cc7832; font-weight: bold;">as </span>plt
date_start = datetime.datetime(<span style="color: #6897bb;">2017</span><span style="color: #cc7832;">,</span><span style="color: #6897bb;">1</span><span style="color: #cc7832;">,</span><span style="color: #6897bb;">1</span>)
date_end = datetime.datetime(<span style="color: #6897bb;">2017</span><span style="color: #cc7832;">,</span><span style="color: #6897bb;">6</span><span style="color: #cc7832;">,</span><span style="color: #6897bb;">30</span>)</pre>
<span style="background-color: #2b2b2b; color: #a9b7c6; font-family: "menlo"; font-size: 9pt;">data = datar.get_data_google(</span><span style="background-color: #2b2b2b; color: teal; font-family: "menlo"; font-size: 9pt;">'AAPL'</span><span style="background-color: #2b2b2b; color: #cc7832; font-family: "menlo"; font-size: 9pt;">, </span><span style="background-color: #2b2b2b; color: #a9b7c6; font-family: "menlo"; font-size: 9pt;">date_start</span><span style="background-color: #2b2b2b; color: #cc7832; font-family: "menlo"; font-size: 9pt;">, </span><span style="background-color: #2b2b2b; color: #a9b7c6; font-family: "menlo"; font-size: 9pt;">date_end)</span><br />
<br />
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Menlo'; font-size: 9.0pt;">short_ma = <span style="color: #6897bb;">5</span>long_ma = <span style="color: #6897bb;">20</span><span style="color: #6897bb;">
</span>data[<span style="color: teal;">'short_ma'</span>] = data[<span style="color: teal;">'Close'</span>].rolling(short_ma).mean()
data[<span style="color: teal;">'long_ma'</span>] = data[<span style="color: teal;">'Close'</span>].rolling(long_ma).mean()
data[<span style="color: teal;">'masig'</span>] = data[<span style="color: teal;">'short_ma'</span>] - data[<span style="color: teal;">'long_ma'</span>]
data=data.dropna(<span style="color: #aa4926;">subset</span>=[<span style="color: teal;">'masig'</span>])
data[<span style="color: teal;">'signal'</span>] = np.where(data[<span style="color: teal;">'masig'</span>] > <span style="color: #6897bb;">0</span><span style="color: #cc7832;">, </span><span style="color: #6897bb;">1</span><span style="color: #cc7832;">, </span><span style="color: #6897bb;">0</span>)
data[<span style="color: teal;">'signal'</span>] = np.where(data[<span style="color: teal;">'masig'</span>] < <span style="color: #6897bb;">0</span><span style="color: #cc7832;">, </span>-<span style="color: #6897bb;">1</span><span style="color: #cc7832;">, </span>data[<span style="color: teal;">'signal'</span>])
<span style="color: #8888c6;">print</span>(data[<span style="color: teal;">'signal'</span>].value_counts())
data[<span style="color: teal;">'returns'</span>] = np.log(data[<span style="color: teal;">'Close'</span>]) - np.log(data[<span style="color: teal;">'Close'</span>]).shift(<span style="color: #6897bb;">1</span>)
data[<span style="color: teal;">'strategy'</span>] = data[<span style="color: teal;">'returns'</span>] * data[<span style="color: teal;">'signal'</span>].shift(<span style="color: #6897bb;">1</span>)
data[<span style="color: teal;">'strategy'</span>].cumsum().plot(<span style="color: #aa4926;">title</span>=<span style="color: teal;">'Cumulative Return'</span>)
plt.show()</pre>
Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com3tag:blogger.com,1999:blog-7565259.post-36998154633976520412016-02-15T15:30:00.005+01:002017-06-29T12:22:48.356+02:00Type-2 Fuzzy Logic in High Frequency TradingIn a recent journal publication we investigate the viability of Type-2 fuzzy systems in high frequency trading. We propose Type-2 models based on a generalisation of the popular ANFIS model (ANFIS/T2). Type-2 models score significant risk adjusted performance improvements over Type-1.<br />
<dt class="label">Benefits of Type-2 models increase with higher trading frequencies.</dt>
<dd><br /></dd>Paper available at:<br />
<a href="http://www.sciencedirect.com/science/article/pii/S0957417416300203">http://www.sciencedirect.com/science/article/pii/S0957417416300203</a><br />
<br />Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com1tag:blogger.com,1999:blog-7565259.post-27541466313049241212014-04-25T08:58:00.001+02:002014-04-25T09:01:08.443+02:00Presenting at the IEEE Computational Intelligence for Financial Engineering and Economics Conference (CIFEr), London 2014.<table class="schedule"><tbody>
<tr valign="top"><td>Enhancing Intraday Trading Performance of Neural Network using Dynamic Volatility Clustering Fuzzy Filter </td></tr>
<tr><td></td><td></td></tr>
</tbody></table>
<br />
<a href="http://ieee-cifer.org/program.html">http://ieee-cifer.org/program.html</a>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-30298942863634927152014-04-25T08:54:00.001+02:002017-06-29T12:21:46.287+02:00Publication on Neurocomputing ...Enhancing risk-adjusted performance of stock market intraday trading with Neuro-Fuzzy systems<br />
<br />
<div class="MsoNormal" style="margin: 0cm 0cm 0pt;">
<span style="font-family: "calibri" , "sans-serif"; font-size: 11pt;"><a href="http://www.sciencedirect.com/science/article/pii/S0925231214004470"><span style="color: blue;">http://www.sciencedirect.com/science/article/pii/S0925231214004470</span></a></span><o:p></o:p></div>
<br />Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-63211953351830880362013-07-28T13:06:00.001+02:002013-07-28T13:11:31.267+02:00What software architects/developers frequently forget ...<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<a href="https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcQM8oqPUViQNXlWu7PPle3h7w_CC9dSZ_cJkrK624Wk-jFJxHRl" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="142" src="https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcQM8oqPUViQNXlWu7PPle3h7w_CC9dSZ_cJkrK624Wk-jFJxHRl" width="320" /></a>When looking at code or architectures, sometimes I get the feel that developers forget that the art of programming, or any other problem solving task, is really the beauty of being able to transform a complex problem into a series of simple, well defined logical steps. Here is what some of the most intelligent people the world has ever seen remind us:<br />
<br />
<span style="background-color: white; color: #181818; font-family: georgia, serif; font-size: 14px; line-height: 18px;">“If you can't explain it to a six year old, you don't understand it yourself.” </span><br />
<span style="background-color: white; color: #181818; font-family: georgia, serif; font-size: 14px; line-height: 18px;">― </span><a href="http://www.goodreads.com/author/show/9810.Albert_Einstein" style="background-color: white; color: #666600; font-family: georgia, serif; font-size: 14px; line-height: 18px; text-decoration: none;">Albert Einstein</a><br />
<br />
<br />
<span style="background-color: white; color: #181818; font-family: georgia, serif; font-size: 14px; line-height: 18px;">“Simplicity is the ultimate sophistication.” </span><br />
<span style="background-color: white; color: #181818; font-family: georgia, serif; font-size: 14px; line-height: 18px;">― </span><a href="http://www.goodreads.com/author/show/13560.Leonardo_da_Vinci" style="background-color: white; color: #666600; font-family: georgia, serif; font-size: 14px; line-height: 18px; text-decoration: none;">Leonardo da Vinci</a><br />
<br />
<span style="background-color: white; color: #37404e; font-family: 'lucida grande', tahoma, verdana, arial, sans-serif; font-size: 13px; line-height: 18px;">"Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it."</span><br />
<span style="background-color: white; color: #37404e; font-family: 'lucida grande', tahoma, verdana, arial, sans-serif; font-size: 13px; line-height: 18px;">- Alan Perlis</span><br />
<br />
<span style="background-color: white; color: #181818; font-family: georgia, serif; font-size: 14px; line-height: 18px;">So, the gist of the story, the role of an excellent developer or architect is not to complicate the simple, but to simplify the complicated. Sometimes people can easily get mislead about the beauty of a solution by admiring its complexity, but really and truly the greatness lies in the ability to simplify a complex problem.</span><br />
<span style="background-color: white; color: #181818; font-family: georgia, serif; font-size: 14px; line-height: 18px;"><br /></span>
<span style="background-color: white; color: #181818; font-family: georgia, serif; font-size: 14px; line-height: 18px;"><br /></span>
<br />
<br />
<br />
<br />
<br />
<br />Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-55135252625279834982013-07-05T19:24:00.003+02:002013-07-05T19:24:42.652+02:00Data Visualisation toolsAt the moment I am looking for some data visualisation tools. Here is an interesting link with some great visualisation tools I met.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://media.netmagazine.futurecdn.net/files/imagecache/featured_main/articles/feature/2012/09/120917visualisationmain.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="174" src="http://media.netmagazine.futurecdn.net/files/imagecache/featured_main/articles/feature/2012/09/120917visualisationmain.jpg" width="320" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<a href="http://www.netmagazine.com/features/top-20-data-visualisation-tools" target="_blank">Web Link</a>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-82759121227229851072013-07-03T23:21:00.000+02:002013-07-03T23:21:29.668+02:00Work Hard, Play Hard!<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dzCx2CNqhT4TIeUb0j3dxq4YhKTDn4NIQCAV2gm3K9AjOvOQ56cOK510Ew5h0X9imIQO7sKCAr3' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dz82XjOmCwz8Nt0EdUuh5B-k3PCkC3JixNWOXYst2J992wp0SFLzh35iK38-B8Fg6M_bIpto1pk2RM' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dxNE1HQm7cv0NQ-Njv-7BhnZtAQ02d2s8Ddbh4zz00IzfzdM9BxKE0mfDcFQKx40hXWqQ--2y44KRc' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
<br />Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-49200233362653030872013-07-01T01:39:00.001+02:002013-07-01T08:48:48.250+02:00Improving risk-adjusted performance profile of intraday trading models with Neuro-Fuzzy techniques and moving average high-frequency price signals<div xmlns="http://www.w3.org/1999/xhtml">
<a href="http://www.essex.ac.uk/ccfea/research/WorkingPapers/2013/CCFEA-WP068-13.pdf">http://www.essex.ac.uk/ccfea/research/WorkingPapers/2013/CCFEA-WP068-13.pdf</a></div>
Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-70454993299551272482010-08-17T02:07:00.003+02:002010-08-17T02:27:05.306+02:00Interfacing C# .Net and R - Integrating the best of both worlds (Part 3) - Trader Desk Example<div style="text-align: justify;">In this third and final part of the series (<a href="http://vvella.blogspot.com/2010/08/integrate-c-net-and-r-taking-best-of.html">Part 1</a>, <a href="http://vvella.blogspot.com/2010/08/interfacing-c-net-and-r-integrating.html">Part 2</a>), I am going to continue shaping the examples in the previous posts by quickly building a small application - a simple Trading Desk application. In the example I will use the same C# and R interface method together with a specific Quantitative Financial Modelling R library called <a href="http://www.quantmod.com/">Quantmod</a>.</div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;"><strong>Example Objective</strong></div><div style="text-align: justify;">In the example the objective is to build a C# Web Form which acts as the main application controller that captures various user option parameters with the main functions being:</div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">1. To automatically kick off an R routine to download data from Yahoo Finance. </div><div style="text-align: justify;">2. To select specific dates of interest</div><div style="text-align: justify;">3. To add or udpate different charts</div><div style="text-align: justify;">4. To add different chart indicators<br />
5. To calculate Period Return Statistics</div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">Most of these functions are provided through the set of R functions exposed by the Quantmod R library. </div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;"><strong>Additions under the hood</strong></div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">Below I am showing some of the salient additions done to the code over and above the code in the previous posts. Since the underlying program structure is in line with the explanations in the previous posts I am not going to delve again into detailed explanations. The code should be pretty self explanatory.</div><br />
<span style="color: #990000;">//My RFacade Class - note the new addition to load Quantmod Library</span><br />
<span style="color: #990000;">public RFacade()</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> rconn = new STATCONNECTORSRVLib.StatConnector();</span><br />
<span style="color: #990000;"> rconn.Init("R");</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"> //load R Quantmod Library</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn("library(quantmod)");</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Get data from yahoo finance</span><br />
<span style="color: #990000;">public void RGetData(String symbol)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = "getSymbols('"+symbol+"',src='yahoo')";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Draw Candle Chart for specific company symbol and date range</span><br />
<span style="color: #990000;">public void RCandleChart(String symbol, DateTime from, DateTime to)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String dateRange = FormatDateRange(from, to);</span><br />
<span style="color: #990000;"> String RCommand = "candleChart(" + symbol +dateRange+",multi.col=TRUE,theme='white')";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><span style="color: #990000;">//Draw Bar Chart for specific company symbol and date range</span><br />
<span style="color: #990000;">public void RBarChart(String symbol, DateTime from, DateTime to)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String dateRange = FormatDateRange(from, to);</span><br />
<span style="color: #990000;"> String RCommand = "barChart(" + symbol + dateRange + ")";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Draw ChartSeries plot for specific company symbol and date range</span><br />
<span style="color: #990000;">public void RTimeSeries(String symbol, DateTime from, DateTime to)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String dateRange = FormatDateRange(from, to);</span><br />
<span style="color: #990000;"> String RCommand = "chartSeries(" + symbol + dateRange + ")";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Create new R grahics window (/device)</span><br />
<span style="color: #990000;">public void RNewChart()</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = "windows()";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Add Indicators</span><br />
<span style="color: #990000;">public void addIndicatorSMA()</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = "addSMA(20)";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">public void addIndicatorDPO()</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = "addDPO()";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">public void addIndicatorBBands()</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = "addBBands()";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Get Mean Return</span><br />
<span style="color: #990000;">public double RGetMeanReturn(String symbol, DateTime from, DateTime to)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String dateRange = FormatDateRange(from, to);</span><br />
<span style="color: #990000;"> String RCommand = "logRet<-diff(log(" + symbol + "))";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;"> RCommand = "mean(logRet" + dateRange + "[,4])";</span><br />
<span style="color: #990000;"> return (double)rconn.Evaluate(RCommand); </span><br />
<span style="color: #990000;">}</span><br />
<div style="text-align: justify;"><br />
</div><div style="text-align: justify;"><strong>How it Looks</strong></div><div style="text-align: justify;">As you can see from the screen shot below, with this approach one can quickly start to mash up interesting applications by benefeting from the rich R platform and the flexible C# front end. </div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">Hope you found this series of articles on C# and R interfacing using R(D)COM interesting ... and please anyone who wishes to continue discussing or collaborate or share experiences just drop me an <a href="mailto:vince.vella@gmail.com">email</a> or post a comment.</div><br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBwuw4JesOPC0n10ePUc0hT-57bVqRbn53pSIuNc8JxKKAN8PWn7ybEsrif-6SbEElufm3zrOYAYyc0aKrkNWl_R3YP7QwvSRMBKaw92TvcAKrBgbLRAp7vnPaY3LdL6qQztmtjg/s1600/TraderExample.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="231" ox="true" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBwuw4JesOPC0n10ePUc0hT-57bVqRbn53pSIuNc8JxKKAN8PWn7ybEsrif-6SbEElufm3zrOYAYyc0aKrkNWl_R3YP7QwvSRMBKaw92TvcAKrBgbLRAp7vnPaY3LdL6qQztmtjg/s400/TraderExample.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Click to Enlarge</td></tr>
</tbody></table>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com7tag:blogger.com,1999:blog-7565259.post-88994237115938293942010-08-13T09:52:00.003+02:002010-08-17T02:13:59.369+02:00Interfacing C# .Net and R - Integrating the best of both worlds (Part 2)<div style="text-align: justify;">This post is a continuation from the previous post (<a href="http://vvella.blogspot.com/2010/08/integrate-c-net-and-r-taking-best-of.html">Part I</a>) focusing on interfacing C# with R using the R (D)COM. In this post I am going to enhance my previous exercise by creating a Facade .Net Class which facilitates access to specific functions in R. </div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;"><strong>Creating the R Facade Class</strong></div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">Creating a Facade Class (or a set of .Net classes) which acts as a .Net wrapper to R functions greatly facilitate the use of R functions and their integration within the .Net programming environment. Below I am showing an excerpt from the class RFacade that I have created in this example.</div><br />
<span style="color: #990000;">using System;</span><br />
<span style="color: #990000;"></span><br />
<span style="color: #990000;">using System.Collections.Generic;</span><br />
<span style="color: #990000;">using System.Linq;</span><br />
<span style="color: #990000;">using System.Text;</span><br />
<span style="color: #990000;">using StatConnectorCommonLib;</span><br />
<span style="color: #990000;">using STATCONNECTORSRVLib;</span><br />
<span style="color: #990000;">using System.Runtime.InteropServices;</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">namespace R</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">class RFacade : IDisposable</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"> private StatConnector rconn;</span><br />
<span style="color: #990000;"> private bool disposed = false;</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"> public RFacade()</span><br />
<span style="color: #990000;"> {</span><br />
<span style="color: #990000;"> rconn = new STATCONNECTORSRVLib.StatConnector();</span><br />
<span style="color: #990000;"> rconn.Init("R"); </span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"> public void Dispose()</span><br />
<span style="color: #990000;"> {</span><br />
<span style="color: #990000;"> Dispose(true);</span><br />
<span style="color: #990000;"> GC.SuppressFinalize(this);</span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"> protected virtual void Dispose(bool disposing)</span><br />
<span style="color: #990000;"> {</span><br />
<span style="color: #990000;"> if (!disposed)</span><br />
<span style="color: #990000;"> {</span><br />
<span style="color: #990000;"> if (disposing)</span><br />
<span style="color: #990000;"> {</span><br />
<span style="color: #990000;"> if (rconn != null)</span><br />
<span style="color: #990000;"> {</span><br />
<span style="color: #990000;"> Marshal.ReleaseComObject(this.rconn);</span><br />
<span style="color: #990000;"> rconn = null;</span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"> disposed = true;</span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"></span><br />
<span style="color: #990000;">.... continued</span><br />
<br />
<div style="text-align: justify;">The RFacade constructor handles the creation of the COM connection and initialisation. As you can notice the above example presents also a slight improvement from the previous post when it comes to closing the connection. The RFacade class implements the IDisposable interface to ensure that all unmanaged resources are released when the wrapper is no longer used.</div><br />
<strong>Setting and Getting Data from R</strong><br />
<br />
<div style="text-align: justify;">Although in the previous post I showed examples how this can be done, in this example I am going to provide methods in my RFacade class which further simplify the setting and getting of data from R. The two RFacade methods are show below. The first method, <em>RGetRandomNormalVector, </em>creates a vector on R which in return is populated by R with a set of random numbers from a Standard Normal Distribution. The second method, <em>RSetVector, </em>provides similar functionality but this time the data is provided by the C# front end. These two methods show how vectors can be passed easily back and forth between the two layers.</div><br />
<span style="color: #990000;">//Function to create an R vector of name [VectorName] and size [size] </span><br />
<span style="color: #990000;">//and generates random numbers from normal distribution</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">public double[] RGetRandomNormalVector (String VectorName, int size)</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = VectorName + "<-rnorm(" + size + ")";</span><br />
<span style="color: #990000;"> double[] data = (double[])rconn.Evaluate(RCommand);</span><br />
<span style="color: #990000;"> return data;</span><br />
<span style="color: #990000;">}</span><span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Function to create an R vector of name [VectorName] with specific data [data] </span><br />
<br />
<span style="color: #990000;">public void RSetVector(String VectorName, double[] data)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> int vectorSize = data.Count();</span><br />
<span style="color: #990000;"> StringBuilder stringData = new StringBuilder(vectorSize);</span><br />
<span style="color: #990000;"> for (int c=0; c<=(vectorSize-1); c++) {</span><br />
<span style="color: #990000;"> stringData.Append(data[c]);</span><br />
<span style="color: #990000;"> if (c < (vectorSize-1) ) { </span><br />
<span style="color: #990000;"> stringData.Append(",");</span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"> String RCommand = VectorName + "<-c(" + stringData + ")";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<br />
<span style="color: #990000;">... continued</span><br />
<br />
<span style="color: black;"><strong>Exposing R functions</strong></span><br />
<br />
At this stage we have everything ready to start using some R functions. In this example I am going to provide some simple functions which provide a good indication of how one can extend this functionality.<br />
<br />
<br />
<span style="color: #990000;">//Function to Calculate Correlation using R cor function</span><br />
<span style="color: #990000;"></span><br />
<span style="color: #990000;">public double RCorrelate(String var1, String Var2)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = "cor(" + var1 + "," + Var2 + ")";</span><br />
<span style="color: #990000;"> return (double)rconn.Evaluate(RCommand);</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Function to plot a simple XY graph using R graphics</span><br />
<span style="color: #990000;">public void RPlot(String var1, String Var2)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = "plot(" + var1 + "," + Var2 + ")";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand.ToString());</span><br />
<span style="color: #990000;">}</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">//Function to Calculate Linear Regression using R lm function</span><br />
<span style="color: #990000;">//Returns a double array, [0] intercept [1] gradient</span><br />
<span style="color: #990000;">public double[] RLinearRegression(String dependentVar, String independentVar)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> String RCommand = "fit <- lm("+ dependentVar + " ~ " + independentVar +")";</span><br />
<span style="color: #990000;"> rconn.EvaluateNoReturn(RCommand); </span><br />
<span style="color: #990000;"> return (double[])rconn.Evaluate("fit$coefficients");</span><br />
<span style="color: #990000;">}</span><br />
<br />
<strong>Using R from C# .Net is easier...</strong><br />
<br />
<div style="text-align: justify;">Once we have our Facade class and also a set of common functions, making use of R becomes much easier from C#. The exerpt below provides examples of method calls from my main Form which accesses a number of exposed R functions via the RFacade object instance <em>RInstance</em>.</div><br />
<span style="color: #990000;">double[] year = new double[] { 2000, 2001, 2002, 2003, 2004 };</span><br />
<span style="color: #990000;"></span><br />
<span style="color: #990000;">double[] rate = new double[] { 9.34, 8.50, 7.62, 6.93, 6.60 };</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">RInstance.RSetVector("year", year);</span><br />
<span style="color: #990000;">RInstance.RSetVector("rate", rate);</span><br />
<br />
<span style="color: #990000;">double corr = RInstance.RCorrelate("year", "rate");</span><br />
<br />
<span style="color: #990000;">RInstance.RPlot("year", "rate");</span><br />
<br />
<span style="color: #990000;">double[] regressionCoeff = RInstance.RLinearRegression("rate", "year");</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">double[] data = RInstance.RGetRandomNormalVector("mydata", dataSize);</span><br />
<br />
<span style="color: black;">This article is continued in a third and final post <a href="http://vvella.blogspot.com/2010/08/interfacing-c-net-and-r-integrating_17.html">Part 3</a>. </span>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com10tag:blogger.com,1999:blog-7565259.post-84056469610477546032010-08-12T14:34:00.009+02:002010-08-13T09:57:20.269+02:00Interfacing C# .Net and R - Integrating the best of both worlds<div style="text-align: justify;">In specific software areas like in quantitative finance or else in other mathematical domains, data centric programming typically requires a good balance between three requirements - (1) a solid platform with rich mathematical/statistical functionality (2) having an easy to use, contemporary, programming environment which permits easy and flexible front end code development and (3) an easy to use interface between the two environments.</div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">In this artcile I am going to explain how such a balance can be attained by using two of the best products in their specific worlds - using the rich <a href="http://www.r-project.org/">R</a> library as the mathematical/statistical component but then interfacing with C# for the front end application design. As an interfacing option I banked on using R (D)COM which provides an easy to use interfacing method which keeps you away from spending hours identifying interfacing problems.</div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">The software required for this tutorial is the following:</div><div style="text-align: justify;">1. R software (<a href="http://www.r-project.org/">download from here</a>)</div><div style="text-align: justify;">2. R (D)COM Interface (<a href="http://cran.ms.unimelb.edu.au/">download from here</a>)</div><div style="text-align: justify;">3. If you dont have a C# IDE you can download Visual Studio Express for free (<a href="http://www.microsoft.com/express/Windows/">download from here</a>)</div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">Once you installed all your software we can start focusing on the small example. Albeit a simple example, the example will show the flexibility that one can attain by interfacing R and C#. In the example we have a C# Windows Form which fires a request to R in order to generate a random data set from a standard normal distribution, calculate some basic statistics, and then display the data and results back on our form.</div><br />
<strong>STEP 1 - Creating the Windows Form</strong><br />
<br />
In Visual Studio create a new Windows Form application. <br />
<br />
<strong>STEP 2 - Add project reference to the R (D)COM library</strong><br />
<br />
<div style="text-align: justify;">From the solution explorer add reference to the R (D)COM library. In the COM components list you typically find the component name listed as "Repository for R COM Server Instances".</div><br />
<strong>STEP 3 - Add library references to your form code</strong><br />
<br />
This is done by adding references as follows:<br />
<br />
<span style="color: #990000;">using StatConnectorCommonLib;</span><br />
<span style="color: #990000;">using STATCONNECTORSRVLib;</span><br />
<br />
<strong>STEP 4 - Setting and Initializing Connection</strong><br />
<br />
<div style="text-align: justify;">Two variables were defined in the form class. The variable rconn of type StatConnector represents the connection to the R COM component. The variable dataSize is used to store the size of the data array that we are going to randomly generate from R.</div><br />
<span style="color: #990000;">private StatConnector rconn;</span><br />
<span style="color: #990000;">private int dataSize;</span><br />
<br />
Both these variables are initialized in the default constructor of my form, in my case named frmMain.<br />
<br />
<span style="color: #990000;">public frmMain()</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;">InitializeComponent();</span><br />
<span style="color: #990000;">dataSize = 100;</span><br />
<span style="color: #990000;">rconn = new STATCONNECTORSRVLib.StatConnector();</span><br />
<span style="color: #990000;">rconn.Init("R");</span><br />
<span style="color: #990000;">}</span><br />
<br />
<strong>STEP 5 - Invoking R commands and displaying results</strong><br />
<br />
The form that I created in my example had the following components:<br />
<br />
1. A listbox which is used to present the random numbers generated from R<br />
2. Two labels which present the Mean and Standard Deviation of the data calculated from R<br />
3. A button which kicks off the process <br />
<br />
The button clicked handler code is presented below:<br />
<br />
<span style="color: #990000;">private void btnGenData_Click(object sender, EventArgs e)</span><br />
<span style="color: #990000;">{</span><br />
<span style="color: #990000;"> rconn.SetSymbol("sdataSize",dataSize);</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"> //Generate Data in R</span><br />
<span style="color: #990000;"> rconn.Evaluate("sdata<-rnorm(sdataSize)");</span><br />
<span style="color: #990000;"> double[] data = (double[])rconn.GetSymbol("sdata");</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"> //Calculate Statistics in R</span><br />
<span style="color: #990000;"> rconn.Evaluate("saverage<-mean(sdata)");</span><br />
<span style="color: #990000;"> rconn.Evaluate("sstdev<-sd(sdata)");</span><br />
<span style="color: #990000;"> double average = (double)rconn.GetSymbol("saverage");</span><br />
<span style="color: #990000;"> double stdev = (double)rconn.GetSymbol("sstdev");</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;"> // Display Data and Results on Windows Form</span><br />
<span style="color: #990000;"> lbDataList.Items.Clear();</span><br />
<span style="color: #990000;"> for (int c = 0; c < dataSize; c++)</span><br />
<span style="color: #990000;"> {</span><br />
<span style="color: #990000;"> lbDataList.Items.Add(data[c]);</span><br />
<span style="color: #990000;"> }</span><br />
<span style="color: #990000;"> lblAverage.Text = "Average: " + average;</span><br />
<span style="color: #990000;"> lblStdDev.Text = "Standard Deviation: " + stdev;</span><br />
<span style="color: #990000;"><br />
</span><br />
<span style="color: #990000;">}</span><br />
<br />
<div style="text-align: justify;">As it can be seen from the above code, the interfacing mechanism is quite simple and intuitive. The only important thing to note is that the COM object returns back general object references which need to be typcasted as required. In the case of the data variable which holds the random data vetor, this is typecasted into a double[] type. In the case of a single return value form average and standard deviation this is typecasted into a double. </div><div style="text-align: justify;"><br />
</div><div style="text-align: justify;">Pretty easy right? I found this method really easy to use and keeps you focused on the problem that you wish to solve, rather than spending hours debugging code. Using C# for your front end application also provides a lot of flexibility when it comes to building world class applications and user interfaces. Testing the method with large amounts of data, running in the hundreds of thousands, also proved to be very efficient.<br />
<br />
A continuation of this article is available in <a href="http://vvella.blogspot.com/2010/08/interfacing-c-net-and-r-integrating.html">Part 2</a></div>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com19tag:blogger.com,1999:blog-7565259.post-46809355885841770522010-04-03T21:02:00.000+02:002010-04-03T21:02:18.743+02:00RunKeeper Shared Fitness | Walking Activity<a href="http://runkeeper.com/user/vvella/activities/3B3Sq7a1xPDU4XDrHjgZ?sms_ss=blogger">RunKeeper Shared Fitness | Walking Activity</a>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-61476154849065239452010-02-15T20:39:00.010+01:002010-08-12T19:50:30.529+02:00Installing OpenCV 2.0.0 on MacOS X 10.6.1 Snow Leopard ( Xcode IDE )I have recently started experimenting with OpenCV. Setting the environment on Windows/Visual Studio is pretty straight forward and works fine. However since my preferred development environment is MacOS X / Unix i preferred to switch environments.<br />
<br />
I must say that the installation went quite smooth, even though on the internet i found a number of posts on a number of issues.<br />
<br />
The steps i followed were the following:<br />
<br />
1. download opencv library from http://sourceforge.net/projects/opencvlibrary/<br />
<br />
2. go to the downloads/opencv-2.0.0 directory<br />
<br />
3. execute ./make_frameworks.sh<br />
<br />
this creates an opencv framework which comes very handy when developing using Xcode<br />
<br />
The process takes quite some time ....<br />
<br />
Once completed using Xcode open the sample project (FaceTracker) for MaxOS x which is found under the opencv-2.0.0/samples directory<br />
<br />
clean, build and off you go ... camera on my mac book pro was as well recognized without problems.Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com1tag:blogger.com,1999:blog-7565259.post-44159310573465200942008-12-29T00:09:00.001+01:002008-12-29T00:09:54.763+01:00Testing my blogposting from iPhone ... Cool<br/><br/><a href='http://maps.google.com/maps?q=35.8784179688,14.3998718262'>Geolocate</a> this post<br/><br/>Posted with <a href='http://lifecast.sleepydog.net'>LifeCast</a>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-62530554018055172112008-07-02T21:40:00.007+02:002008-08-05T00:00:42.800+02:00SPSS Regression using Matrices<span style="font-size:78%;"><span style="font-size:100%;"><span style="font-family:arial;">The SPSS syntax below does several linear regression calculations (model parameter values, sum of squares, F value, F test, R, Rsquare, t tests, etc) using matrices ... just change the data file and initialization part as required and off you go ......<br /><br /></span></span><br /><br />/* ####### DATA FILE ################################################################################### */<br /><br />FILE HANDLE data /NAME='c:\data\ratty2.sav'.<br /><br /><br />/* ####### INITIALIZATION AND DATA MATRIX ############################################################## */<br /><br />MATRIX.<br /><br />COMPUTE N=21. /* no of data rows */<br />COMPUTE IND_VARS=3. /* no of independet variables */<br /><br />GET Y /FILE=data /VARIABLES =size. /* puts y-variable into column matrix called Y */<br />GET X1 /FILE=data /VARIABLES =t,z2,z3 /* put independent variables into a matrix called X1 */<br /><br />/* ########## MATRIX CALCULATIONS #################################################################### */<br /><br /><br />COMPUTE j=make(N,1,1). /* Create column vector of 1's */<br />COMPUTE X={j,X1}. /* joins column of 1's in front of X1 columns */<br />COMPUTE XT=TRANSPOS(X). /* XT is the transpose of X */<br />COMPUTE XDX=XT*X. /* X'X */<br />COMPUTE IXDX=INV(XDX). /* (X'X)^ -1 */<br />COMPUTE BETA=IXDX*XT*y. /* BETA= ( (X'X) ^ -1)X'y */<br />COMPUTE BETAT=TRANSPOS(BETA).<br />COMPUTE YT = TRANSPOS(Y).<br />COMPUTE JT=TRANSPOS(J).<br />COMPUTE MODELSS=BETAT*XT*Y-(JT*Y)**2/N.<br />COMPUTE ERRSS=YT*Y-BETAT*XT*Y.<br />COMPUTE TOTALSS = YT*Y-(JT*Y)**2/N.<br />COMPUTE MMS=MODELSS/IND_VARS.<br />COMPUTE ERRMS=ERRSS/(N-IND_VARS-1).<br />COMPUTE F=MMS/ERRMS.<br />COMPUTE FP=1-FCDF(F,IND_VARS,N-IND_VARS-1).<br />COMPUTE VAR=IXDX*ERRSS.<br />COMPUTE VAR=VAR/(N-IND_VARS-1).<br />COMPUTE SE=SQRT(DIAG(VAR)).<br />COMPUTE TVAL=BETA/SE.<br />COMPUTE TVALF=TVAL&**2.<br />COMPUTE PVAL=1-FCDF(TVALF,1,N-IND_VARS-1).<br />COMPUTE TAB = {BETA,SE,TVAL,PVAL}. <br /><br />COMPUTE RSQ= MODELSS/TOTALSS.<br />COMPUTE ADJRSQ=1-ERRMS/TOTALSS*(N-1).<br /><br />/* ########### OUTPUT ######################################################################################## */<br /><br />PRINT BETA.<br />PRINT X.<br />PRINT Y.<br />PRINT XT.<br />PRINT XDX.<br />PRINT IXDX.<br />PRINT MODELSS.<br />PRINT ERRSS.<br />PRINT TOTALSS.<br />PRINT MMS.<br />PRINT ERRMS.<br />PRINT F.<br />PRINT FP.<br />PRINT VAR.<br />PRINT SE.<br />PRINT TAB.<br />PRINT RSQ.<br />PRINT ADJRSQ.<br /><br />END MATRIX.</span>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-33598325959687942712008-06-26T23:12:00.008+02:002008-06-26T23:33:54.325+02:00Java Design PatternsToday i had the pleasure to present a session at the monthly Java User Group Malta meeting ... its always a pleasure to meet other Java Developers in our community and aspiring students. Click on the post tile to link to the presentation ...<div><br /></div><div>Regards and good luck guys ... hope to meet again for other sessions.</div><div><br /></div>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com1tag:blogger.com,1999:blog-7565259.post-44313815538639659882008-06-22T20:30:00.001+02:002008-06-26T23:34:59.887+02:00Java Web Application Properties<div style="text-align: justify;">One of the most common application requirements is usually to create a central place where to keep application properties or application settings. These properties are normally settings or general properties that are applied to the application during runtime. A number of options exist, including creating string properties on your application server and accessing them as JNDI resources. However one of the still most practical approaches is to keep a properties config file with a simple <span class="Apple-style-span" style="font-style: italic;">propertyName</span><property><span class="Apple-style-span" style="font-style: italic;">=propertyValue</span> <property>type of structure. The example below is an example of an Application Config class that loads the application properties from a file named config.properties which is stored in the root of the source packages (root package) and hence automatically deployed with the application jar file during the build process.</property></property><br /></div><property><property><div><br /></div><div><!--StartFragment--> <p class="MsoPlainText"><span lang="EN-US"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">import java.io.IOException;</span></span></span></p><p class="MsoPlainText"><span lang="EN-US"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">import java.io.InputStream;</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">import java.util.Properties;</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">import java.util.logging.Level;</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">import java.util.logging.Logger;</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><o:p><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">public class ApplicationConfig {</span></span></o:p></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=" ;font-family:Georgia;"><span style="mso-spacerun: yes"><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style=" "><span class="Apple-style-span" style="font-size:small;"> </span></span></span></span><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style=" "><span class="Apple-style-span" style="font-size:small;">private static Properties properties;</span></span></span></span></span></span></span></p><p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=" ;font-family:Georgia;"><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style=" "><span class="Apple-style-span" style="font-size:small;"> private static ApplicationConfig config;</span></span></span></span></span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">static {</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">config = new ApplicationConfig();</span><span class="Apple-style-span" style=" ;font-family:Georgia;"><span style="mso-spacerun: yes"><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style=" "></span></span></span></span></span></span></span></p><p class="MsoPlainText"><span lang="EN-US"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style=" ;font-family:Georgia;"><span style="mso-spacerun: yes"><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style=" "><span class="Apple-style-span" style="font-size:small;"> </span></span></span></span><span class="Apple-style-span" style=" ;font-family:'courier new';"><span class="Apple-style-span" style=" "><span class="Apple-style-span" style="font-size:small;">}</span></span></span></span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">private ApplicationConfig ()</span></span><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">{</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">try {</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">InputStream in = this.getClass().getClassLoader().getResourceAsStream("config.properties");</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">properties = new Properties();</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">properties.load(in);</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">} catch (IOException ex) {</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">Logger.getLogger(ApplicationConfig.class.getName()).log(Level.SEVERE, null, ex);</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">}</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">}</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">public static ApplicationConfig getApplicationConfig () {</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">return config;</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">}</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">public String getProperty (String propName) {</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">return properties.getProperty(propName, "Null");</span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">}</span></span><span style="mso-spacerun: yes"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;"> </span></span></span></span></p> <p class="MsoPlainText"><span lang="EN-US"><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">}</span></span></span></p> <!--EndFragment--> </div><div>An example config.properties file might contain something like:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">applicationTitle=This is my application</span></span></div><div><br /></div><div>An example test client (which might be a servlet, jsp, etc) can use this class as follows:</div><div><br /></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">ApplicationConfig c = ApplicationConfig.getApplicationConfig();</span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><span class="Apple-style-span" style="font-size:small;">String title = c.getProperty("applicationTitle");</span></span></div><div><span class="Apple-style-span" style="font-family:'courier new';"><br /></span></div></property></property>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com2tag:blogger.com,1999:blog-7565259.post-11487742077243100042008-05-18T20:41:00.007+02:002008-05-18T21:11:00.326+02:00Glassfish and MySQL - Connection Issue<div style="text-align: justify;">Hi all<br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I recently downloaded the new Netbeans 6.1 including the Glassfish v2 package to replace my previous versions. Once I moved my application to the new platform I had to temporarily deploy the application database from Oracle to MySQL to test on my local machine. My application is built on JavaEE and for the persitence layer I am making use of data access objects (using Toplink) fronted by a set of session beans. After porting the database for testing I started getting the following error:</div><div style="text-align: justify;"><br /></div><div style="text-align: left;"><span class="Apple-style-span" style="font-family:'courier new';">Internal Exception: java.sql.SQLException: Error in allocation a connection. Cause: No PasswordCredential found</span></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">This problem doesn't occur during development within Netbeans (hence all jdbc resources and Persistence units work fine from within netbeans) but occurs on deployment on Glassfish.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">From some google searches i found that this problem existed for some time with previous versions but still exists with the latest installation of Netbeans/Glassfish including a bundled MySQL. A solution for this problem is to add the password property manually on the glassfish server -> jdbc -> database connection pools from the Administration Console. Once this is done the error will disappear. To test that this problem exists you can try to ping the database from the msqlpool Administration screen and you notice that on installation the ping gives this error.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Although its a small thing I believe that with the new Netbeans 6.1/Glassfish v2/MySQL bundle Sun can fix this setting by default and hence save some debugging time! </div><div><br /></div>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com3tag:blogger.com,1999:blog-7565259.post-57459937074846249052008-03-30T20:20:00.002+02:002008-03-30T20:39:22.576+02:00Mac OS X - first impressionsI recently had to buy a laptop for my personal use. To the surprise of my closest colleagues I have bought a Mac Book Pro. My first impressions are excellent, money well spent. From the OS perspective I found java pre-installed and all the nice unix tools like ssh which i have to use to connect to my linux servers available in the 64 bit unix base OS. Remote access using VNC works perfect as well. Setting up glassfish and netbeans was again very easy to do without an problems - performance was excellent, better than i ever seen on windows platforms for sure - an experienced IT person can immediately feel the solid OS thats running under the hood. When it comes to the leopard interface one cant comment, definitely one of the sleekest and most user friendly environments i've seen so far. Just to conclude my short note, for the java developers -> its definitely a nice environment to work in, an x platform with the slickest environment around.Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-43181275973772947612007-07-10T23:39:00.000+02:002007-07-11T23:54:05.429+02:00Java Persistance<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnTjka6bCEo6XQzzk3y9qomvDjPVag6-DNxfAxeYgiT6uoGyDBQnS8IVs1JjGRs0NnrPcidXxAdLsaLNupvXdxwZ5XAsu2kmqvyh59v3PdH7tEYb3sRn1seO6EJuN7l5-tGXzhBw/s1600-h/persist1.JPG"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnTjka6bCEo6XQzzk3y9qomvDjPVag6-DNxfAxeYgiT6uoGyDBQnS8IVs1JjGRs0NnrPcidXxAdLsaLNupvXdxwZ5XAsu2kmqvyh59v3PdH7tEYb3sRn1seO6EJuN7l5-tGXzhBw/s320/persist1.JPG" alt="" id="BLOGGER_PHOTO_ID_5085689718891615522" border="0" /></a><br /><div style="text-align: justify;">Recently I explored a bit how one can easily make use of the new Java Persistance API. The example below shows how one can very quickly create and search records from a Derby database table called "Course".<br /><br />To keep things simple and at the same time examine exactly the code required for such a task I created a simple console application from my netbeans 5.5.1 IDE. The new persistance API is much lighter than the previous version and was taken out from the Enterprise API thus making it possible to use it from simple java applications. Once I created my console app I added a new "Entity class from Database" to the project. A dialog is displayed and the connection chosen must be previously defined from the Runtime->Databases section in Netbeans. In the dialog I as well chose the option to "Create Persistance Unit" using the Toplink Library and named it "PUnit".<br /><br />Once one goes through these simple steps automatically Netbeans creates an Entity class named "Course" with getter and setter methods for each individual table field.<br /><br />Then I went into my Main class generated initially for my console application and modified as follows:<br /><br /></div><br /><span style="font-size:78%;"><span style="font-family:courier new;">package javaapplication4;</span><br /><br /><span style="font-family:courier new;">import javax.persistence.EntityManager;</span><br /><span style="font-family:courier new;">import javax.persistence.EntityManagerFactory;</span><br /><span style="font-family:courier new;">import javax.persistence.Persistence;</span><br /><br /><span style="font-family:courier new;">/**</span><br /><span style="font-family:courier new;">*</span><br /><span style="font-family:courier new;">* @author Owner</span><br /><span style="font-family:courier new;">*/</span><br /><span style="font-family:courier new;">public class Main {</span><br /><br /><span style="font-family:courier new;">private EntityManagerFactory emf;</span><br /><span style="font-family:courier new;">private EntityManager em;</span><br /><span style="font-family:courier new;">private String PERSISTENCE_UNIT_NAME = "PUnit";</span><br /><br /><span style="font-family:courier new;">private void initEntityManager() {</span><br /><span style="font-family:courier new;"> emf = Persistence.createEntityManagerFactory(PERSISTENCE_UNIT_NAME);</span><br /><span style="font-family:courier new;"> em = emf.createEntityManager();</span><br /><span style="font-family:courier new;">}</span><br /><br /><span style="font-family:courier new;">private void closeEntityManager() {</span><br /><span style="font-family:courier new;"> em.close();</span><br /><span style="font-family:courier new;"> emf.close();</span><br /><span style="font-family:courier new;">}</span><br /><br /><span style="font-family:courier new;">private void create(String code, String title, short days, String details) {</span><br /><span style="font-family:courier new;"> em.getTransaction().begin();</span><br /><span style="font-family:courier new;"> Course c = new Course();</span><br /><span style="font-family:courier new;"> c.setCode(code);</span><br /><span style="font-family:courier new;"> c.setTitle(title);</span><br /><span style="font-family:courier new;"> c.setDays(days);</span><br /><span style="font-family:courier new;"> c.setDetails(details);</span><br /><span style="font-family:courier new;"> em.persist(c);</span><br /><br /><span style="font-family:courier new;"> em.getTransaction().commit();</span><br /><span style="font-family:courier new;">}</span><br /><br /><span style="font-family:courier new;">private void searchTitleByCode (String code) {</span><br /><span style="font-family:courier new;"> Course c = (Course)em.createNamedQuery("Course.findByCode").setParameter("code",code).getSingleResult();</span><br /><span style="font-family:courier new;"> System.out.println(c.getTitle());</span><br /><span style="font-family:courier new;">}</span><br /><br /><span style="font-family:courier new;"> /** Creates a new instance of Main */</span><br /><span style="font-family:courier new;"> public Main() {</span><br /><span style="font-family:courier new;"> }</span><br /><br /><span style="font-family:courier new;"> /**</span><br /><span style="font-family:courier new;"> * @param args the command line arguments</span><br /><span style="font-family:courier new;"> */</span><br /><span style="font-family:courier new;"> public static void main(String[] args) {</span><br /><span style="font-family:courier new;"> Main m = new Main();</span><br /><span style="font-family:courier new;"> m.initEntityManager();</span><br /><span style="font-family:courier new;"> m.searchTitleByCode("ABC");</span><br /><span style="font-family:courier new;"> m.closeEntityManager();</span><br /><span style="font-family:courier new;"> }</span><br /><br /><span style="font-family:courier new;">}</span><br /><br /></span><br /><div style="text-align: justify;"><span style="font-size:100%;">The simple code above shows how one can create and search for a Course record. For searching I used the "Course.findByCode" named query which was created automatically in my Entity Class (by browsing in the Entity Class one can easily identify this query together with other generated named queries). To run the application you have to as well add the Java DB Library with the project libraries. Result = Very neat approach to DB persistance!</span><br /></div>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com2tag:blogger.com,1999:blog-7565259.post-67846405454817742312007-06-21T22:33:00.000+02:002007-06-21T22:38:34.971+02:00Swapping two variables without using a third<div style="text-align: justify;">I came across an interesting algorithm ... the normal way how programmers swap values between two variables is by using a temporary third variable ... temp = x, x = y, y = temp. An interesting trick can actually do the same process without having to use a third variable, and this is done by using the XOR operator. Below I am showing how this can be done in java:<br /></div><br /><br /><span style="font-family: courier new;">public static void main(String[] args) {</span><br /><span style="font-family: courier new;"> int x = 15;</span><br /><span style="font-family: courier new;"> int y = 20;</span><br /><span style="font-family: courier new;"> System.out.println(x+" "+y);</span><br /><span style="font-family: courier new;"> x ^= y;</span><br /><span style="font-family: courier new;"> y ^= x;</span><br /><span style="font-family: courier new;"> x ^= y;</span><br /><span style="font-family: courier new;"> System.out.println(x+" "+y);</span><br /><span style="font-family: courier new;"> }</span><br /><br /><br />Try it out !Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com1tag:blogger.com,1999:blog-7565259.post-57103471448241684242007-06-20T18:40:00.000+02:002007-06-20T18:43:48.488+02:00Java Tech Day in Malta!Finally the first ever Sun Java Tech Day in Malta. It will definitely be an excellent opportunity for the Java community in Malta to hear leading speakers from the US. Details on the event can be found at <a href="http://mt.sun.com/javaday">http://mt.sun.com/javaday</a>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-86313021015977744242007-06-16T14:25:00.000+02:002007-07-11T00:31:35.070+02:00Worth noting on the radar ... O3Spaces<div style="text-align: justify;">A product called O3Spaces Workplace is gaining momentum in the collaboration software field. The product is being positioned as a direct competitor to MS SharePoint Office Server focusing on document management and collaboration. The open edition should be available at some point during 2007 both on Windows and Linux platforms. From the available demos and documentation it surely seems to be an excellent competitor for SharePoint bringing in as well nice integration with OpenOffice tools. For further info (and even a vmware image demo) you can find more info at <a href="http://www.o3spaces.com/">http://www.o3spaces.com</a></div>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-78989795644158326652007-06-10T22:57:00.000+02:002007-07-11T00:31:58.481+02:00Setup for my development environment<div style="text-align: justify;">Finally I managed to find the time to complete a proper setup at home. As my server I used Ubuntu 7.04 as a base to host my Glassfish app server, Postgres and other utils. Mainly i use it as a deployment box and for my development i still use my Netbeans on my XP laptop managing all the rest through ssh/psftp over wireless access mainly. Compared to other distributions i found Ubuntu extremely easy to install and driver recognition is one of the best i have ever seen compared to other Linux distributions. Surely sudo and apt-get commands make life extremely easy to install and setup services. In conjunction with the autodeploy function on Glassfish this makes the deployment of war/ear files really easy ... anyways really happy with the environment, the plan is to soon start exposing some utilities from my linux box over the net, will keep you posted.</div>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0tag:blogger.com,1999:blog-7565259.post-23592592524459192502007-06-07T19:37:00.000+02:002007-07-11T00:32:29.402+02:00Lacking creativity ...<div style="text-align: justify;">Yesterday I visited MCAST since I was one of the judges examining student programming projects. What really struck me is the fact that the majority of projects, although technically sound, didn't really explore any new areas or ideas ... a situation of more of the same. Would have expected much better from young students who were given free hand on the topics or areas to focus on. Although not a teaching expert myself, i think that it is highly important that teachers and the teaching model itself introduce creative thinking methods as part of the IT students curriculum.</div>Vincent Vellahttp://www.blogger.com/profile/08290318498931566170noreply@blogger.com0