Big Memory: An Interview with Terracotta CEO Amit Pandey

first_img8 Best WordPress Hosting Solutions on the Market I did an interview this morning with Terracotta CEO Amit Pandey about the fascinating new dimensions of in-memory data and its use for search. How excited I was when I learned that the public relations person had been transcribing the interview. It felt like a Mad Men moment!It turned out well enough to include in its entirety. Pandley covers the fast evolving world of big memory. The term is catchy isn’t it? Amit certainly gives a pitch here but we should expect that in a format like this. The interview is insightful in it shows how fast data can now be accessed and what it means for the new world of real-time intelligence.Alex Williams: So tell me what you’re seeing in the market these days?Amit Pandey: Our most interesting thing is — I had briefed you on BigMemory, right? It has surpassed our expectations, which were high, in terms of how useful it is and how many people want it. Everything from the name itself, which stirs customer imaginations–the choice of name was quite good because it resonates with people’s desire to use more memory. You’re getting servers that are much larger with more memory. Most CIO offices feel an imperative to make data more accessible in real-time. Most people know that really the only way to do that is put it in-memory, where the applications can reach it very quickly. The name grabs people’s imaginations, they dig in and find it is what they thought it was, and they start doing proof of concepts and we go from there. What we’ve seen is that with this product, with BigMemory, we have much more data on its performance now, and we’re able to do generally about 4-10x the level of density of other products and even compared to our own older, non-BigMemory products, in terms of how much hardware and how many servers it takes to achieve the level of in-memory data that customers want.I’ll give you an example. There’s a large bank that had its data distributed across 30 servers to achieve the amount on in-memory data they need. With BigMemory, they were able to bring it down to two servers. That was a huge win for them because the cost of administration suddenly dropped. The complexity of running 30 servers isn’t even comparable to the complexity of running two.Williams Okay, so that’s an example of that 4-10x density?Pandey: Yeah. In this case it was a 15x increase. But theoretically it’s possible to do that use case on one server. We did tests with Cisco’s UCS server and we’ve been able to run 350 gig on a single UCS server. That’s bigger than a lot of people’s entire database. In the past, I would say 6-8 gigs per application instance was about the most you could do.Williams: You said that’s bigger than most people’s database?Pandey: 350 gigs? Sure. There are databases that go into terabytes and petabytes, but I would say the average database is probably around that size or smaller. So because in one instance of an application we can get that kind of in-memory storage, it’s a huge win. We’re seeing that’s very exciting to people because it simplifies their architecture and makes it a lot more elegant. I think you remember the reason for the 6-8 gigabyte limit was that the Java memory manager is very poor at handling bigger sizes, and BigMemory by-passes that. So that’s just a quick recap.In general, the whole concept of doing things in-memory and in real-time is big on everyone’s agenda–real-time analysis, etc. The natural step for us was if we can put this much data in memory, people will want to do things with it. They’ll want to do searches and queries of that data to find out what their customers are doing, what their customers need, and so forth, so real-time analysis of that data is critical. We’re releasing a native search capability for Ehcache which basically lets you search as much memory as you would need.Here’s an example of the kinds of searches customers are doing: we have a SaaS customer that does logistics for fast food restaurants. Normally, they run reports against a database–there are two issues with that. One is that they were not getting real-time data because they would batch stuff and write it back to the database and run their reports on a four hour basis or at the end of the day. What customers really need is to find out at any given moment how many hamburger buns they have in inventory and how many have been used up so they know where they are and can do real-time management and lower costs and make sure they don’t run out, etc.To do that against the database was very slow because it meant going across the network to the disk. It also meant the database was very overloaded and it meant spending tons of money expanding the database to get this done. To work against the database was taking them about 35-40 seconds to do some of these reports. But BigMemory with Search dropped their times from 35-40 seconds down to less than half a second. Williams: So what were they were they trying to understand?Pandey: Search was for inventory items like hamburger buns or other food items, and they were trying to figure out in real-time how much has been used right now and how much is left. They needed real-time analysis of data and doing real-time analysis on a database was both very slow and very expensive. Williams: So instead of having to do that against a database, they can now use in-memory to do that a lot more efficiently?Pandey: Exactly, because the data is right there. You don’t need to make a round trip to go across the network to the database. The cost is a lot lower because they’d have to buy a lot more database licenses to achieve the speeds they need.Williams: It’s interesting how that can affect the supply chain, too.Pandley: Sure, it can. Their customers won’t tolerate those kinds of search speeds. So the logistics company was looking for another solution. Without BigMemory, this company would probably have had to spend a lot of money upgrading to a really expensive solution like Oracle Exadata or something like that.Williams: Where is this all going? Pandey: Our customers use it for all kinds of things: we have travel reservation systems running on Ehcache, websites, online gaming systems, back-end medical patient records. In any places where you need to do a quick search and query of what your customers are doing in real-time, you can do that. You could do a search and say, how many people are currently logged in and playing my game who are 25 years old and live in Oklahoma, because I want to do a promotion in Oklahoma for those people right now. You can do that with a database, but it would be very slow and very expensive. With in-memory data you can do that really fast, and target those people quickly.One thing I do want to make clear, Alex: we’re not saying we are replacing analytics in the database. We’re not providing all the heavyweight reporting capabilities that business intelligence tools offer today for databases and we’re not doing all the analytics. But what we are providing is a very simple, powerful, lightweight search where you can do real-time analysis of customer behavior and things like that. Over time we’ll make it a richer reporting set.We’re working with BI and other vendors to provide hooks so that they can run their stuff against ours. It will become richer over time. So right now, we provide a simple lightweight thing that’s extremely useful for real-time analysis but you couldn’t really say it’s a business analytics tool yet because those have been developed over the years and the term “analytics” is loaded. We’re very careful to use the term “real-time analysis”. Over time, in the next 3 to 5 years, I see this getting richer. You’re already seeing all these companies (SAP, etc) talking about merging analytical and transactional together in one architecture. What we are doing is essentially that, we’re taking baby steps toward that.Williams: With tablets available, you can see this data visually, that has an added impact.Pandey: Yeah. The great thing is if you put search capability in the application, it’s sort of independent of the platform that uses it–it could be a phone, tablet, etc. Your platform can be leveraged by any of these devices. Obviously, mobile devices would be a big part of that.Williams: Thanks for your time!(Photo: Amit Pandey, CEO, Terracotta) alex williams Related Posts A Web Developer’s New Best Friend is the AI Wai…center_img Tags:#cloud#cloud computing Top Reasons to Go With Managed WordPress Hosting Why Tech Companies Need Simpler Terms of Servic…last_img read more

2010: The year of treasure hunters

first_imgThe first Indian to welcome US President Barack Obama at Mumbai airport on November 6 was then Maharashtra chief minister Ashok Chavan. He was sacked minutes after Obama left India because of his direct involvement in the Adarsh Housing Society scam in which flats meant for Kargil heroes and widows,The first Indian to welcome US President Barack Obama at Mumbai airport on November 6 was then Maharashtra chief minister Ashok Chavan. He was sacked minutes after Obama left India because of his direct involvement in the Adarsh Housing Society scam in which flats meant for Kargil heroes and widows were virtually stolen by powerful politicians, bureaucrats and defence officers who abetted construction on disputed army land. Chavan’s relatives owned three flats in Adarsh. He was confident that his mentors – a guru, Sri Sathya Sai Baba of Puttaparthi, and Rahul Gandhi – would save him. Chavan was self-delusional. It was the judiciary that forced the political class to move. On December 22, a division bench of Justice B.H. Marlapalle and Justice U.D. Salvi hearing the case noted, “Everybody who was supposed to clear a file has been gifted (a flat in Adarsh).”Adarsh was not the only scandal in the Year of the Treasure Hunter. The total loot of the public exchequer in 2010 is estimated to exceed Rs 4 lakh crore. An analysis reveals that real estate is corruption’s biggest cash cow. According to realtors, 60 to 70 per cent of the price of property transactions in all major Indian cities is in cash. A finance ministry report of December 2009 states: “Of the Rs 4,500 crore seized in income tax raids, around Rs 2,000 crore involves land. Politicians are earning money per sq ft of the floor space index awarded to construct buildings. This is a result of commercialisation of politics,” says former Mumbai High Court Judge B.G. Kolse-Patil. The CBI discovered that it was Chavan, who as state revenue minister in 2000 cleared the society’s ownership conditions to admit civilians along with defence personnel, thereby opening up a real estate gold-mine. “The Congress is the fountainhead of corruption in the country,” says BJP leader Rajnath Singh. advertisement Raja as telecom minister flouted rules to grant spectrum licences to select companies. Though Chavan’s declared income in 2009 was Rs 24.61 crore, in reality he owns assets worth hundreds of crores. He bent laws to get flats for politicians, bureaucrats and army officers in a housing society meant for Kargil heroes and widows.The BCCI accuses Modi of financial bungling and his relatives of making money through IPL bids.                      In 2010, allegations of corruption – referred to by Wharton academic Jitendra Singh as “a kind of shadowy market” – were made against the spectrum of the national establishment. Politicians like Chavan, B.S. Yeddyurappa, Suresh Kalmadi and Mulayam Singh Yadav; senior army officers like former army chief General Deepak Kapoor and Major General R.K. Hooda stole the headlines of the year with a glittering cast of friends, mentors and hangers-on from the media and bureaucracy.Chavan’s personal assets run into hundreds of crores of rupees. In his hometown Nanded, Chavan Inc owns dealerships, real estate worth crores, sugar mills and educational institutions. He insists that his commercial ventures are legal and he has done nothing wrong. For a politician who declared his net worth as Rs 24.61 crore in 2009, INDIA TODAY investigations revealed that the estimated worth of assets held by Chavan and his family amount to much more. “The politicians in the entire country are corrupt. Why talk only about Maharashtra?”says Kolse-Patil. Shiv Sena leader Uddhav Thackeray says, “We are facing a challenge to cure the disease of corruption.” Chavan, however, is defiant. “The investigations will prove my innocence,” he says. The CAG report pointed out massive corruption in the CWG and claimed crores were stolen by officials and contractors. The Karnataka chief minister refused to resign over charges that government land was allotted to family members. This land was returned after a judicial inquiry was set up. When he was the chief minister of Uttar Pradesh between 2004-2007, foodgrain meant for the poor was diverted.The country has got used to the idea of corrupt politicians. It was shocked to learn that officers of the rank of general and admiral were involved in Adarsh. “Here is a man whom we trusted, here is a person on whose orders we were ready to go to battle,” said Chief of Staff General V.K. Singh about his predecessor General Kapoor. In his application for a Rs 80-lakh apartment, General Kapoor lied that his annual income was Rs 2.8 lakh while when it was actually around Rs 7.2 lakh. When this application was turned down, the then Maharashtra chief minister Vilasrao Deshmukh used his arbitrary powers to clear it. Adarsh is not the only example of Kapoor’s real estate adventures. The Haryana Government gave him a Gurgaon plot for Rs 36 lakh with a lock-in payment period of five years – a knockdown price for land valued at Rs 7.5 crore. In Sukna, West Bengal, army land was allegedly given to builders at throwaway prices. The military court indicted two army officers for this deal – one of them was Kapoor’s staff advisor Lt. General Avdesh Prakash. Kapoor reportedly used his clout to save Prakash from a court martial. The armed forces have earned the unsavoury nickname, the “Harmed Forces”.advertisementReal estate besmirched the reputation of Karnataka’s Chief Minister B.S. Yeddyurappa as well. In June 2010, Yeddyurappa was preparing for a spotlight moment, ordering a bespoke Italian suit from Bangalore fashion designer Prakash Lamba as a change from his usual white safari suit to preside over the two-day Global Investors Meet in Bangalore from which he expected to raise at least $66.3 billion as investments for Karnataka. In private, Yeddyurappa was taking decisions that would blow up in his face. On November 17, 2010, Karnataka Lok Ayukta Santosh Hegde announced there was enough evidence to establish a prima facie case against Yeddyurappa.The Opposition released documents – around 2,000 of them – alleging that Yeddyurappa flouted rules to get prime land worth Rs 500 crore allotted to family members; his sons BJP MP B.Y. Raghavendra and Vijayendra became beneficiaries of state largesse after their father became the Karnataka deputy chief minister in 2006. Raghavendra was allotted a government plot in Bangalore under the MP’s quota and paid Rs 10 lakh for the land whose market value was over Rs 2 crore. Yeddyurappa also denotified the Bangalore Development Authority (BDA) land which his sons and son-inlaw Sohan Kumar allegedly grabbed at throwaway prices. Yeddyurappa’s daughter Umadevi was allotted two acres of government land to set up a BPO. Some of his family members were allegedly allotted five acres in Shimoga to build a hospital, and two acres for a factory in Bangalore. In March 2010, 11 more acres in Bangalore worth crores were denotified in Vijayendra and Raghavendra’s favour against the advice of the BDA and the Urban Development Department. Later, his kin returned the land following a judicial inquiry. Five previous chief ministers gave similar deals to their relatives including Dharam Singh and H.D. Deve Gowda’s son H.D. Kumaraswamy who is yet to return the land allotted to him.”The names of my family members are being dragged into imagined corruption cases to cover up their (Opposition’s) own misdeeds,” says Yeddyurappa, responding to the Opposition’s charges of corruption.The Opposition also alleges that the chief minister took a bribe of Rs 20 crore from the Bellary-based South West Mining Ltd; supporting documents claim that the money was allegedly deposited into the bank accounts of Raghavendra, Vijayendra and Sohan Kumar. Graft FilesAccording to corruption watchdog Global Financial Integrity,Rs 20,79,000 crorewas transferred out of the country illegally between 1948 and 2008.That money could repeat the UPAI farm loan waiver scheme 35 times.************************************************In 2008, black money in India amounted to Rs 28 lakh crore. Just half of that could retire India’s foreign debt.The remaining would be enough to give every Indian Rs 14,000.advertisement************************************************The Swiss BankingAssociation Report of 2008 reveals that Indians hold $1,891 billion in deposits. That’s about Rs 8,50,950 crore; in Rs 1,000 notes laid end to end, this amount would girdle the globe more than 37 times.************************************************With the 2G money the Government lost thanks to Raja,8,800 crore solar power plants generating 1 MWeach could have been put up nationwide, enough to resolve our electricity shortage many times over.************************************************The estimated amount paid by Indians as bribes to the legal system is Rs 2,360 crore.If land was exploited by the corrupt, the skies were open to political plunder as well. On November 14, the DMK, unable to withstand the heat from a belligerent Opposition, a hostile Congress and an angry media, ordered Andimuthu Raja to resign as telecom minister. Raja was accused of favouring select telecom companies- Swan Telecom, Unitech, S-Tel, Datacom Solutions, Idea, Spice, Systema, Tata and Loop-in acquiring 2G spectrum licences in a valuation game which is estimated to have cost the country a notional Rs 1.76 lakh crore. “As a lawyer, I will abide by the law, not evade it,” Raja said. His slogan during the 2009 Lok Sabha polls was “Oru kilo arisi oru rupa, oru hello 50 paisa” (One kg rice at one rupee, a hello on the phone at 50 paise).The Niira Radia tapes exposed how much his hellos cost the country. Veteran Congress leader of Tamil Nadu EVKS Elangovan says “Karunanidhi must’ve been aware of things.” In power, there is no free lunch. So what did the telecom companies give Raja and Radia in return of the favours? The CBI and the ED are investigating a money trail that leads to the Cayman Islands, the Virgin Islands, the Bahamas and Mauritius. “Raja represents a front for a number of undesirable organisations. He is a front for anti-national agents,” says Subramanian Swamy, Janata Party president. There may be political exaggeration in the accusation.Raja stuck to his chair for 400 days after the scam surfaced. A CBI official claims that the minister used this time to manipulate or destroy evidence. No incriminating documents were found when the agency raided his house on December 8. According to one source, documents were also destroyed at Radia-owned Vaishnavi Communications’s office in Delhi. Employees had burned thousands of documents to prevent them from falling into the hands of the CBI. “The Government should constitute an Integrity and Ethics Commission urgently to check corruption and initiate stringent action against corrupt practices,” says ASSOCHAM President Dilip Modi.The climate of venality spawned a variety of hunters who sought to seize what they could. Public anger began to mount. When Suresh Kalmadi, the most visibly tainted face of the 2010 Commonwealth Games, walked into a Delhi restaurant on a late October weekend, he was surrounded and heckled by diners who called him a thief. The CBI and Central Vigilance Commission (CVC) estimate that the 66-year-old Congress Rajya Sabha MP caused a loss of Rs 5,000-8,000 crore of public money through financial and administrative malfeasance. In 2003, the Indian Olympic Association estimated the Games would cost the country Rs 1,620 crore. By 2010, the bill had risen to Rs 11,500 crore. With other additional expenditure, the Games expenditure was estimated to be around Rs 30,000 crore, making the 2010 Commonwealth Games the most expensive sporting event ever. CBI sources says that of the Rs 665 crore spent on overlays, around Rs 200 crore was stolen. Following the CVC report on financial misdemeanors, the CBI raided the IOA offices and arrested close Kalmadi aides T.S. Darbari, Sanjay Mahindroo and treasurer Jayachandran.Some of the statistics are stunning. The infrastructure cost for the CWG was Rs 2,904 crore but Rs 5,503 crore was released. Where did the extra money go? The CVC investigated 16 Games-related projects totalling Rs 2,477 crore and found that corrupt officials and contractors tampered with initially discounted tenders. The organising committee inflated bills by at least 30 per cent. The CBI raided Kalmadi’s residences in Delhi and Pune. “I am innocent till proven  guilty. I have not taken any decision alone,” Kalmadi says.Many more careers and reputations tumbled as other skeletons of corruption fell out of hidden closets. In Mumbai, private financier Rajesh Sharma, chief of Money Matters (India), and senior bank officials were arrested in a bribes-for-loans affair. The arrests have made corporate India uneasy. Though the licence raj has ostensibly ended, lobbyists are still needed for project clearances. “We need to be clear about what is defined as corruption. In an economy where competition and enterprise are driven by individual endeavours, and there is no strong government hand, these things flourish,” says P. Shankar Raman, executive V-P (finance), L&T. Following these arrests, the market value of LIC Housing Finance shares plummeted from Rs 10,159 crore to Rs 2,365 crore. Investors lost Rs 2,300 crore in a day. “There are enough regulatory bodies in the country but they don’t have the teeth to enforce regulations,” says Richard Rekhy, CEO, KPMG, one of the largest of audit, tax and advisory services firms in the world.There was one unusual instance of corruption. While Adarsh, the CWG and other scams largely involved powerful cabals, Ketan Desai, president of the Medical Council of India, was the lone ranger of graft. His modus operandi was to ask for bribes in return of government recognition for private medical colleges. After the CBI arrested him for accepting Rs 2 crore as payola for one such deal, raids conducted in Delhi, Punjab, Mumbai and Ahmedabad revealed that Desai and his family owned 10 houses, two commercial properties in Mumbai and gold worth Rs 39 lakh.During the Adarsh hearing, Justices Marlapalle and Salvi asked, “Why has no FIR been registered except the one on the missing files in the case? Don’t tell us committees will be formed to conduct inquiries. Committees are formed and dissolved, nothing happens.” Nothing happens because a corrupt system closes its ranks to protect its own. India remains stuck in the purgatory of corruption.- With Nivedita Mukherjee, Kiran Tare and Nirmala Ravindranlast_img read more