ENM Choosing Algorithms 3

we got to get a bunch of experts with using particular algorithms to make sure we are working on a XXXXXX our results by how well we are using models and we take XXXX a bunch of algorithms that are being used and published to do this same thing. so we used that couple of implementations about XXXXX networks, meaning same approach implemented by two different people two different groups using two different bits of code essentially XXXX below XXX traits, classification trees XXXXXXX the details of the most important arrange of different approaches that view the world in very very different ways we then make predictions these are 2 species of Proteasaea we make predictions on the present and future climate scenarios it's actually a scenario for the 2030s and then we calculated the predicted range gained or lost under those climate scenarios so basically we took a model which was threshold elementXX we talked about threshold element, that is to say we have binary predictions of suitable and not suitable versus continuous probability surface or suitability surface and then we estimated what the percentage of present day range and future day range so we took two scenarios, one of which was XXX full dispersal meaning that any area that is occupied in the future was predicted to occupied whereas we also did a notice XXX scenario where we set the only areas that were going to occupied in the future those areas kind of overlapped with the current distribution so we are going to assume that if we were predicting some other area within South Africa in the future but the species isn't currently present there, the species didn't have the opportunity to actually get there I'm not going to go to anymore details on that because Enrique is going to talk about these things within the context of the locations later on the only important point now is to say that we were able to quantify the predicted range gains and losses so that you only get a loss of course when you assume that there is no dispersal ability OK, you are going to get this last XXXXX no dispersal ability the only important point I want to emphasize now is that you get very very different predictions from different models, so this particular species again one model predicts that it could expand massively 300% expansion in its range area under the future climate scenario where asusing the same dispersal assumptions we had other models for a XXXX actually example, the predicted and almost complete contraction of that species range almost complete collapse of the suitable area that was available for that species so hugely different results we did this cluster analysis that kind of proves methods together and it was interesting to see some of the more complex algorithms ended up grouping some of the more simple algorithms XXXXX quite complex in many ways proves XXX predictions is very simple climate envelope range the point I was going to make here XXXXXXXXXXXXXXXXXXX is that the model predicts the model selection doesn't always but can have a very big impact on your predictions so you need to be very careful about selecting your models justifying models and increasingly testing more than 1 model so that you can make qualitative statements that aren't subjective aren't dependent on the actual model that was predicted ideally presenting the work in thesis or to a government agency or to a publication you might be able to say my conclusions aren't dependent on an arbitrary choice about which algorithm I use so you need to have good justification for the algorithms that you've used, you need to understand how they are functioning and you need to be able to say that my results or conclusions is not dependent on the actual algorithm just want to emphasize as this is a really key paper. Lead author is XXXXXXXX that is XXXX and XXX in 2006 where they did similar to what we've discussed they didn't look at climate change scenarios, they just looked at a whole bunch of Algorithms and they took a whole bunch of data sets and runs the model based on 2 axis of model performance so we are going to talk about what these axis mean in detail tomorrow but basically think lower performance, higher performance and lower performance to higher performance in your better models should be plotting up here and your poor models should be plotting down here it's a really key paper, it's one that I recommend to you to look at it's neat things for example the data that was used to build the models that was independently collected from the data that was used to evaluate the models so what you see again is that the models don't all predict the same XXX some rankings in terms of which models are able to perform better. the general conclusion here was the model that were able to fit more complex responses, some of these new methods XXX regression trees XXXX you can fit these very complex response surfaces are better able to predict niches and distributions than some of these more simple approaches like BIOCLIM domain Enrique is going to talk to you about BIOCLIM in just a minute when mines done We are going to have some important XXXX to that in terms going to expand on this a little bit later today or tomorrow, but there is a very valid and serious concern that these evaluation approaches and the way that this study was done in effect rewards XXXXX I'm going to detail that in more context and come with some more examples later XXXXXX the course, but I would apply to it as an important part of this field and an important study also important XXXX in terms of the potential that the results can be influenced by an out XXXXXXXXXXXXXXX and the final that I would like to mention which is another important kind of area of research is the idea of concensus modelling or onsong modelling OK, if the models perform differently how do we deal with that? XXXX key work done particularly by XXXX and colleagues what we should be doing if building a whole bunch of different models and then looking at patterns of consistency across those models so conceptually this is a neat paper, I'll give you the records in a moment suppose this is the region of the world that we are interested in in where we are right now, suppose these are that kind of envelopes or predictions from a whole range of different models this might be a BIOCLIM model or XXXX model or your network, a XXXX model of your regression tree these are the predictions from a range of different models the argument here is that, what we might do is we might get some sort of consistency among the XXXXXX so this area shown here in yellow, it might just be those areas where all of the models predict so this is a kind of concensus that all of the models think we should be predicting this is a crucial area here in the distribution and the niche of the species and we might say this area with a light blue here is the area where I would say half of the models predicted or atleast half of the models predicted and this area where broader prediction here might be kind of any model predicts this might be the area where any of the models if you look at all of them atleast one of the models predicts that this would be a suitable area. so this is a kind of like trying to tackle this issue of model on certainty and different performance of different methods you might lookk at this with more complication, you might start saying this is where all of my models predict this is where 4 out of 5 predict, 3 out of 5 predict 2 out of 5, 1 out of 5 predict so these kinds of ways of looking at predictions if you are predicting many different models take that XXXX to the full extreme you might be able to predict enough different methods to XXX the different ways you can really build up all kinds of probability XXXX so that you pick out kind of most XXXXXX most models predict to kind of lower probability errors so this is an important stream of kind of literature that we wanted to flag for you it's worked XXXX XXXX on trying to apply these approaches it's not XXXX XXXXX to say not necessarily the overall solution you should definitely be using concensus approaches. it's an important thing to read about and consider that you might want to do I also think that another important way of looking at this is to say well I don't want to run 20 or 100 different methods to build up a probability surface but I do want to use more than one method so I can look at the uncertainty in my method so we should be able to go from this week with atleast 3 different methods that you understand that you can XXXXXXXXX carefully, that you can go away and start running more than one method, suppose I XXX XXXX algorithm and then look at the uncertainty that comes from applying more than one algorithm I strongly advise you to look at more than 1 algorithm this is an important approach so XXXX XXX refer to it to kind of through the gates of XXXXX let's just get all approaches we can XXXXXX XXXXXXXX and consistency across them well, there's a balance between that versus selecting and very carefully understanding a small group of models that you can understand very well that you very carefully parameterize and you understand therefore what the predictions are meaning so there's a balance that we wanted to emphasize one model little uncertainty and model 11 gets better and there are decent approaches for concensus models but pretty XXXXX to look at that's just by the way the introduction I was trying to give you, some general considerations into things to think about and read about, there are just some of the key efforts this is the South African study that basically says that there can be uncertainty from using different methods this is the XXXXXXX et al. paper that I've been encouraging you to have a look at, XXXXX more about later and here is the XXXX and XXXXXX paper on some more forecasting which is supposed to grow in XXX from now but I think XXXXXXXXX in starting point understand these concensus or approaches