Comments
Building the Big-Data Analytics Architecture
You must login to participate in this chat. Please login.

A very useful (at times inspiring) presentation - thank you Mark !

Prospector

Thank you Mark, I got a lot insights from your webinar.

Prospector

what open source visualization softwares are most friendly

Prospector

The Capability Model in Action - Ahh deep understanding of the data and related business - this is, in my opinion the one aspect that is most resisted - people don't want to know that much and it hurts processes.

Prospector

Trying to implement the Analytic Cycle is problematic in an office that is used to the Requirement=based approach – it is a whole new paradigm.

Prospector

So in building this predictive modeling project - I am assuming that this an interdepartment project

Prospector

Veracity of data - Is it reliable - This is a very problematic aspect of our data.  You have two maybe three departments involved in data collection and storage and we have seen times when one or the other departments have modified the data to their specific needs (not discussing before hand with others).

Prospector

Velocity - In my opinion this should be discussed upfront for any new data-analytics process.  Executives always want things asap but is this really necessary?  Discussion and reflection on what is needed and when - things might not be needed asap.

Prospector

Thanks again A2 and Mark - excellent discussion, sorry I didn't catch it live.

Prospector

among organizations that move to a data centric discipline, interesting to see the rationalizations between build vs buy when it comes to platforms.

Prospector

curious to know explore thoughts on transforming an existing organization and staff into a data centric organization.

Prospector

excellent information - thank you Mark

Prospector

Good differentiation, thank you.

Prospector

Glad to see realisim, as well removing some of the mytique, in defining the skills needed for a Data Scientist/Worker, even from a Data Scientist: advanced expertise and experience in at least these three disciplines: Information Technology, Business Expertise from the internals of the organization, and Advanced Analytics as well as the desire to do they work required (acquiring education along the way). This puts realism into the real skills that can be acquired to do the work as well as realistic salaries to match the skills.

Prospector

The best infrastructures start with the best people, process (goal setting, new structures...) and the best technology.

Prospector

Planning ahead of time for excess capacity makes sense so that analytic teams can deliver ahead of expectations makes money available for future projects a little easier.

Prospector

Some of the approaches today miss the critical point you made that "Analytics is about bringing the best information to the Point of Decision", it seems implicit in the very nature and reason for doing analytics in the first place.

Prospector

Mark, thank you for some refreshing new thoughts on how to get to real results faster than what most consutants are presenting; the requirements approach doesn't work and should be replaced by a new paradigm.

Prospector

Thanks mark for the wonderful and informative discussion

Data Doctor

what about software architecture, whether its suitable for other offerings too

Data Doctor

data arhitecture, whether its a hardware indipendent design

Data Doctor

sorting algorithms are more important

Data Doctor

Thanks for the info.

Prospector

great convos today

Prospector

what is the benefit of ORacle bigdata ?

Prospector

"Information has a time value": good to know that

Prospector

Thanks for the information.

Prospector

These are usually prety good.

Prospector

I really liked this one.

Prospector

As we wrap up today, don't forget to register for next week's session, Powering Predictive Analytics With Big-Data, if you haven't yet done so. See you there!

Blogger

can't wait for the next one

Prospector

Really good presentation.

Prospector

Yes, thank you so much for the information Mark.  I certainly have learned alot !   Have a great day everyone and see you on the boards !

Blogger

And, don't forget to check out Mark's All Analytics posts! He always has interesting insight to share! Have a great day everybody

Blogger

Thank you Mark and for everyone that helped to make this a cool chat. I got to get back to work now take care everyone :)

Prospector

Thanks again for joining us everyone

@Thanks Mark once again for your response. Awesome presentation. I found it really valuable !!

Prospector

jray88 - I think there will always be a need for smart people who know how to work with data and communicate across disciplines

Blogger

Mark, thanks so much for being our lecturer today. I know I learned a lot, and it seems others have as well. 

Blogger

@rinmaka - I like the part time education option - it allows you to continue to gain experience and immediately apply what you learn in the real world

Blogger

Yes we do ask and are looking for data scientists. Buts that mostly up to the HR department. After they screen them according to our specs then they give us a list of those that made the cut and we take it from there

Prospector

@Mark - To me, Big Data seems like a permanent need now.  Do you agree that the need for data scientists will probably last 10 years or more, or even grow, even if not as "hot" as now?

Prospector

@Beth - yes - data visualization is key in at least two respects - understanding your data, and then telling the story of your data to non-data people

Blogger

@Mark. Thank you for your response. which one is recommended? Full time Vs Part time options for picking up data scientist skills?

Prospector

@rinmaka - I prefer the commercial tools, but work with what you have.  If you can parse text, build a term document matrix, and run an SVD, you can do text analytics

Blogger

Any other last questions before we say our goodbyes?

Blogger

We're almost out of time, but I do want to ask one last question, and that's on data visualization. When you talk to people who are or want to be data scientists are you now adding that into your disucssion as a desired skill set?

Blogger

@Mark, which option would be better, using Opensource tools vs Commercial tools when you want to work on unstructured data?

Prospector

@Beth - it *is* fun - I have to say.  The best part for me is when I can show a business person a solution to a problem they didn't think could be solved.  I get a real charge out of that!

Blogger

Mark, I'm guessing you have a lot of fun at work -- especially since starting to use HPA and playing with big-data!

Blogger

jray88 - I hope it won't be over by then (depends on how long you are planning on taking to finish, LOL)  But I don't see this slowing down anytie soon

Blogger

@Doug!  Thanks for tuning in!  I typically use a supervised approach, i.e these data are associated with X, and these aren't...but there is a lot of manual work in manipulating the data to allow text analytics to distinguish the patterns

Blogger

@beth - I do have a good relationship with IT, but that was built over time - it didn't start that way  =)

Blogger

Will the Big Data revolution be over by the time I get my 2cd degree?? ;)

Prospector

Great job, Mark.

I need to start rummaging in the unstructured data. :)   Any easy ideas, or references you can point to, on how to separate "has conditon x" in text data from "rule out condition x" ?

(Mark used to work in the same hospital I do, so this question will make immediate sense to him.)

Doug Dame - Gainesville FL

Prospector

@benmoreland - the relationship with IT and with the various "silos" of data can be very challenging.  I tyr to overcome those by using the value proposition - if we can break down those silos we can get to significant ROI

Blogger

@rnimaka - You can certainly learn by gaining experience and learning on your own, but the credibility that comes with the credentials counts for a lot

Blogger

Yes, there are a couple of vendors who are coming out with more complete SQL interfaces that completely bypass mapreduce and implement the full SQL standard (or close to it)

Blogger

mnorth -- good question. Mark, it sounds like you've got a good relationship w/ IT, having been in that world?

Blogger

Enjoyed your presentation today.  Curious that you didn't discuss the organization issues (data fiefdoms) that exist that must be broken down. That seems to me to be the hardest problem.

Prospector

@Mark In one of sllides, you have mentioned about the advanced expertise in varied fields like IT, Business and Advance analytics and you have mentioned few examples of people having multiple degrees. Is there any other alternate way where one can gain expertise on all these domains without going to college, if yes, how and where does one have to start?

 

Prospector

I've been thinking about the Meta's recent blog post about Kiss & Make Up with IT in the context of your presentations today.  Big-Data Analytics Architecture often is going to pull the IT folks in, and they often don't share your vision and see having to support hardware or software changes as a burden.  How do I get IT to cooperate with me?  They often see my Big Data projects as creating more work for them.

Blogger

Have you had the opportunity to play around with any of those bypass options? And what's the attraction of those?

Blogger

However, everyone seems to think Hadoop will be central to big data going forward

Blogger

Most of those are becoming options - some vendors are coming out with products that bypass Hive, Pig, and MapReduce

Blogger

Thanks, Mark -- informative presentation!

Prospector

Thank you Mark.  This was very helpful.

Prospector

Thanks, Mark -- informative presentation!

Prospector

Thank you for your informative insights!

Prospector

Thanks, Mark -- informative presentation!

Prospector

Mark, when we think about big-data analytics, are we always involving Hadoop, Hive, Pig, MapReduce -- or are those just options?

Blogger

Are there any questions I can try to answer?

Blogger

The presentation is archived on he site

Wow. Awesome presentation by Mark

Prospector

Thanks, Beth - I hope everyone found some value

Blogger

Whoops-- there you are. Great presentation!

Blogger

How can I obtain the audio of this presentation, it was very good and to the point!

Prospector

Thanks Mark, Beth and Noreen     Lot's of great information !

Blogger

Mark, have you joined us yet?

Blogger

Hello all - thanks again for tuning in

 

Blogger

There is always a trade-off

Blogger

@GeorgiaCougar   Great point,  not sure how this problem can be overcome, even with dedicated lines, there has to be some latency somewhere.....

Blogger

Hadoop? Not here...

Data Doctor

From a network perspective: 

Priority goes to Voice over IP phones, then Video over IP, then important data, then not important data. This could fit in the 3rd place but converged network traffic will cause quick access to be an ongoing challenge.

Data Doctor

I am not using Hadoop

Blogger

And some of us don't like writing Java code at all!

Blogger

@Noreen   Well I would think you have to makes sure your network is optimized from servers to network throughput, figure how you are going to draw data from sources the most efficiently, and then port this data into a program that can turn this data into something useful quickly. The goal of course is to beat the competition to the punch !      No small task ...and I am sure I have forgotten something.

Blogger

good example. good info

Data Doctor

dw4dw -- one great takeaway from today's session for you then. Yay!

Blogger

I agree.  I will use this card example when explaining MPP to the business.

Prospector

@alihashmi   Nothing yet,  I have been so busy with my normal responsiblities which is mainly tech related, I have not had a chance to think about extending my skill set in this area.  Which is something I plain to do in the comming months.

Blogger

I love his card example. Really helps explain the value of MPP.

Blogger

I have the Business background ( MIS), the IT  experience but I need to fill in the  advanced analtytics that is required.  The hardest part IMO.

Blogger

@LouisWatson - what qualifications program are you currently taking?

Prospector

What do the rest of you think? How do you know you are a data scientist?

How do you really know when you are a Data Scientist? Things change so fast... 

Data Doctor

bloomey, Louis -- what do you have, what are you missing?

Blogger

I am getting there!!

 

Prospector

I am almost there .....with the Data Scientist qualifications !

Blogger

Who considers him or her self a data scientist?

Time is key. The days of a device user waiting a few seconds are gone. People move on til they find something with <10ms responce

Data Doctor

Turning this data around quickly into some useful information is vital....and really the difference between a company that effectively deals with Big Data versus those who just want to say they do.....

Blogger

Louis, how do you do this?

@Noreen  I think the ability to rapidly acquire large dataset is major, especially if Big Data is the goal.   The thought given to capturing and storing this data cannot be underestimated.

Blogger

Does that goal sound similar to what others strive to achieve?

Blogger

Thanks for joining us SteveUWS

Thought provoking.  Thank you.

Prospector

As Mark notes:

You need capability to

• Rapidly acquire large datasets from a variety of sources

• Explore, join, and transform those datasets

• Make sense of unstructured data

• Apply many different modeling techniques

Do all of this in time to make a difference

Great point, about analytics being about bringing the best information to the point of decision

Blogger

@Noreen   They certainly are, I love them !

Blogger

GeorgiaCougar,

Time has become essential, hasn't it?

Looking at slides... good stuff

Data Doctor

CaryW30, are you researching any solutions?

The slides are excellent, aren't they?

The challenge is we need the ability to search through all images all video all text stored data in less than a second and compare it to a current image, video, text data... on the fly if you will... 

Data Doctor

@Beth, Noreen   I really love the Slides, really helps to keep everything clear, in sense moving from unstructured to structed data.

Blogger

We so far have been fortunate in that we are able to handle the demand for big data but again thats now. I am very worried about the future because along with our clients we now are seeing a rise of workers byod and they are taking these home. And when they do that we cannot secure it or know who has access to it.

Prospector

This is some great stuff

 

Prospector

How do you keep up with the data in your enterprises or organizations? Or are you struggling?

OP has made good points...

Prospector

absolutely mnorth. lots of other examples, too.

Blogger

Walk into a store and the store says Good afternoon _____ your name... Would you like to buy another _____ or _____

Data Doctor

@Beth   So right  !   Can't beat em,   join em !    ;  )

Blogger

Unstructured data can also be in image form, such as maps, photos, videos, etc.

Blogger

Good point, re: electricity, Louis!

Blogger

Louis, just wait till big-data analytics becomes commonplace! You won't be able to do ANYTHING without somebody knowing.

Blogger

With electric cars coming on strong....oil is not what it used to be ...."Data should be our new electricity."

Blogger

Big-Data is watching us

Data Doctor

My every move is BORING

 

Man, Do I really love my every move tracked.....

Blogger

@David1165-- yes, Mark is speaking now.

Blogger

Is the class on?

 

Prospector

Hadoop?  or other options.

Statistician

Audio is working:-).

Prospector

As Mark gets started, I'm curious -- where are you all in terms of your big-data efforts?

Blogger

Audio seems fine here also

Prospector

working for me as well ....

Blogger

audio is working here

Data Doctor

someone wake up Mark

 

Prospector

Hello everyone..

Prospector

How big is your data? Looking forward to this...

Data Doctor

Hello World. All white and freeeeeeeeeeeeeeeeezing cold....

Prospector

Hi everybody. We'll be starting the audio portion of today's lecture, "Buliding the Big-Data Architecture," at the top of the hour, at which time the audio interface will appear. If you have difficulty, please follow the troubleshooting guide above.

Blogger
Salut to all.
Data Doctor

another hour until we get to build. Lego Blocks ready

Data Doctor

Good morning rnvry

Data Doctor

overall pretty good webinars

Prospector

Some are better than others

Prospector

hope good slides

Prospector

Most have some redeeming value

Prospector

hope this one is good

Prospector

Sounds interesting

Prospector

Greetings Earthlings

 

Prospector

Good morning.

 

Prospector

should be interesting hope to attend

Prospector

Good Morning! Looks like I am a day early, or is it a dollar short?

Prospector

Great to be here!

Prospector

Hello everyone.  Have a nice weekend.

Prospector

I hope this telecast works out. :)

Prospector

Looking forward to this chat

Prospector

Looking forward to hearing this discussion.

Prospector

Free t-shirt !!

 

Prospector

hope to attend...

Prospector

looking forward to it :)

Prospector


INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +